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I. 


INTRODUCTION  AND  BACKGROUND 


Communities  of  Practice  (CoPs)  are  groups  of  individuals  who  learn  and  perform 
better  through  interaction  as  a  result  of  a  shared  passion  or  concern  [1],  Through  social 
networking,  sharing  knowledge,  and  collaboration,  members  of  CoPs  often  outperform 
the  average  individual  performing  the  same  job  assignment  [2].  This  thesis  makes  the 
assumption  that  the  Navy  would  benefit  from  designing  a  community  of  practice  (CoP) 
for  Navy  programmers  and  developers,  in  order  to  strengthen  the  cyber  workforce  and 
support  Navy  cyber  missions  and  objectives.  Fostering  a  CoP  specific  to  the  practices  of 
programmers  and  developers  may  enhance  cyber  mission  success. 

This  research  focuses  on  the  potential  benefits  gained  from  designing  a  CoP  for 
programmers  and  developers,  and  a  methodology  to  cultivate  such  a  community  in  the 
Navy.  The  term  programmer  refers  to  individuals  engaged  in  both  software  and  firmware 
programming  activities.  The  term  developer  refers  to  individuals’  engaged  in  a  system’s 
lifecycle  that  involves  code.  Additionally,  the  term  Navy  programmers  and  developers  is 
not  exclusive  to  Navy  individuals  performing  these  duties  in  an  official  capacity,  but  it 
also  extends  to  all  Navy  individuals  who  engage  in  these  practices  in  an  unofficial 
capacity  or  as  a  hobby.  This  chapter  introduces  the  stimulus  for  this  thesis  and  the 
structure  of  the  Navy  cyber  community.  It  also  outlines  the  research  scope  and 
methodology,  and  summarizes  the  primary  research  questions.  The  benefits  from  this 
study  follow  in  turn,  and  the  last  section  outlines  how  this  research  is  organized. 

A.  TODAY’S  NAVY  CYBER  COMMUNITY 

U.S.  Fleet  Cyber  Command/U.S.  Tenth  Fleet  is  the  Navy’s  “authority  for 
cyberspace  operations,  as  well  as  the  operational  authority  and  capability  provider  for 
Information  Operations  and  cyberspace  operations”  [3].  As  Commander,  Fleet  Cyber 
Command,  they  operate  as  the  Navy  component  commander  to  U.S.  Cyber  Command.  As 
Commander,  U.S.  Tenth  Fleet,  they  direct  naval  operations  including:  network  operations 
and  defense,  information  operations,  service  cryptologic  component  operations,  fleet  and 
theater  operations,  and  research  and  development  [3].  The  vision  of  Fleet  Cyber 
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Command  is  to  “conduct  operations  in  and  through  cyberspace,  the  electromagnetic 
spectrum,  and  space  to  ensure  Navy  and  Joint/Coalition  freedom  of  action  and  decision 
superiority  while  denying  the  same  to  our  adversaries”  [3].  The  Fleet  Cyber  Command’s 
and  Tenth  Fleet  mission  statements  amplify  their  goals  and  objectives: 

The  mission  of  Fleet  Cyber  Command  is  to  serve  as  central  operational 
authority  for  networks,  cryptologic/signals  intelligence,  information 
operations,  cyber,  electronic  warfare,  and  space  capabilities  in  support  of 
forces  afloat  and  ashore;  to  direct  Navy  cyberspace  operations  globally  to 
deter  and  defeat  aggression  and  to  ensure  freedom  of  action  to  achieve 
military  objectives  in  and  through  cyberspace;  to  organize  and  direct  Navy 
cryptologic  operations  worldwide  and  support  information  operations  and 
space  planning  and  operations,  as  directed;  to  execute  cyber  missions  as 
directed;  to  direct,  operate,  maintain,  secure,  and  defend  the  Navy’s 
portion  of  the  Department  of  Defense  Information  Networks  (DODIN);  to 
deliver  integrated  cyber,  information  operations,  cryptologic,  and  space 
capabilities;  to  deliver  a  global  Navy  cyber  common  operational  picture; 
to  develop,  coordinate,  assess,  and  prioritize  Navy  cyber, 
cryptologic/signals  intelligence,  space,  information  operations,  and 
electronic  warfare  requirements;  to  assess  Navy  cyber  readiness;  and  to 
exercise  administrative  and  operational  control  of  assigned  forces. 

The  mission  of  Tenth  fleet  is  to  serve  as  the  Numbered  Fleet  for  Fleet 
Cyber  Command  and  exercise  operational  control  of  assigned  Naval 
forces;  to  coordinate  with  other  naval,  coalition  and  Joint  Task  Forces  to 
execute  the  full  spectrum  of  cyber,  electronic  warfare,  information 
operations  and  signal  intelligence  capabilities  and  missions  across  the 
cyber,  electromagnetic  and  space  domains.  [3] 

The  mission  statements  discuss  Fleet  Cyber  Command  and  Tenth  Fleet 
operational  areas.  The  mission  statements  do  not  state  who  is  tasked  with  completing  the 
mission.  Therefore,  it  is  important  to  define  which  personnel  carry  out  these  operations. 

Within  the  Navy,  there  is  an  undefined  formal  and  an  informal  cyber  workforce. 
This  thesis  defines  the  term  “formal  cyber  workforce”  to  include  all  personnel  (Navy 
Officers,  Navy  Enlisted,  Navy  Civilians,  and  Navy  Contractors)  who  work  in  billets 
associated  with  cyberspace  operations.  In  most  instances  the  formal  cyber  workforce  is 
involved  in  completing  the  tasks  outlined  in  the  mission  statements  above.  However,  the 
mission  statement  of  Fleet  Cyber  Command  does  not  encompass  the  full  spectrum  of 
cyber  risks  the  Navy  faces.  For  example,  Navy  platform  information  technology  such  as 
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industrial  control  systems,  aviation  mission  computers,  and  weapon  systems  are  not  in  the 
Fleet  Cyber  Command  mandate,  but  are  essential  to  Navy  operations.  The  uniformed  and 
civilian  workforce  that  deals  with  these  systems,  from  design  to  operation,  are  an 
important  informal  component  of  the  Navy  cyber  workforce.  Therefore,  we  also  define 
the  term,  “informal  cyber  workforce”  to  include  all  personnel  (Navy  Officers,  Navy 
Enlisted,  Navy  Civilians,  and  Navy  Contractors)  not  serving  in  a  cyberspace  operations 
billet,  but  performing  duties  associated  with  that  of  a  programmer  or  developer  delivering 
computer  dependent  capability  to  the  Fleet. 

Defining  the  formal  and  informal  cyber  workforces  is  important  to  this  thesis, 
because  we  make  the  assumption  that  greater  collaboration  between  the  two  workforces 
will  enhance  the  execution  of  Fleet  Cyber  Command’s  specific  missions,  and  strengthen 
the  ability  to  defend  cyber  systems  outside  the  limited  Fleet  Cyber  Command  mandate. 

B.  SCOPE  AND  METHODOLOGY 

The  thesis  focuses  on  the  potential  benefits  to  be  gained  by  establishing  a  Navy 
CoP  for  programmers  and  developers  that  bridges  the  formal  and  informal  cyber 
workforces.  However,  the  scope  of  this  thesis  also  includes  the  way  the  Navy  engages 
with  industry  partners. 

A  broadened  scope  that  includes  industry  partnerships  is  necessary  because  a 
Navy  CoP  with  a  purely  inward  looking  focus  would  prevent  knowledge  diffusion  and 
mentorship  opportunities  that  may  enhance  the  Navy’s  capability  to  execute  cyber 
missions. 

The  methodology  employed  in  this  research  is  human-centered  design  via  a  social 
interaction  prototype,  which  allows  us  to  leam  and  summarize  the  needs  of  individuals  in 
order  to  design  an  efficient  and  sustainable  CoP  for  Navy  programmers  and  developers. 
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C.  RESEARCH  QUESTIONS 

The  primary  research  question  explored  in  this  thesis  is: 

•  How  might  the  Navy  benefit  from  a  Navy  CoP  for  programmers  and 
developers? 

Secondary  questions  explored  in  this  thesis  include: 

•  What  are  CoPs  and  why  are  they  important? 

•  What  are  key  characteristics  of  successful  CoPs? 

•  What  successful  communities  exist  within  the  military?  What  successful 
communities  external  to  the  military,  are  used  by  military  organizations? 

•  Who  might  be  sponsors,  champions,  and  stakeholders  for  a  Navy  CoP  for 
programmers  and  developers? 

•  What  are  design,  design  thinking,  and  human-centered  design? 

•  How  might  the  Navy  design  a  CoP  for  programmers  and  developers  that 
incorporate  key  characteristics  shared  by  existing  successful  communities? 

•  What  design  method  might  be  utilized  to  design  a  CoP? 

•  What  additional  research  or  prototyping  might  help  the  Navy  further  the 
design  and  sustainment  of  a  Navy  CoP  for  programmers  and  developers? 

D.  BENEFITS  OF  STUDY 

Organizations  that  have  instituted  well-designed  and  effective  CoPs  have 
experienced  many  benefits,  including  improved  collaboration,  implementation  of  a 
shared  framework,  improved  dialogue,  stimulated  learning,  captured  and  diffused 
knowledge,  improved  organizational  skills,  and  the  generation  of  enhanced 
knowledge  [4].  These  benefits  are  highlighted  as  part  of  the  value  proposition  for  a  Navy 
CoP  for  programmers  and  developers.  The  result  of  this  research  adds  to  our 
understanding  of  how  to  design  a  CoP  using  key  characteristics  shared  among  successful 
communities. 

Furthermore,  this  thesis  provides  Tenth  Fleet  with  recommendations  to  design  a 
Navy  CoP  for  programmers  and  developers  in  order  to  make  the  Navy  more  combat 
effective. 
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E.  ORGANIZATION  OF  STUDY 


Chapter  II  introduces  fundamental  concepts,  theories,  and  practices  of  CoPs  to 
better  understand  benefits  gained  from  community  engagement  across  an  organization.  It 
also  introduces  common  terms  used  throughout  the  thesis.  Finally,  it  makes  a  case  for 
why  the  Navy  needs  a  CoP  for  programmers  and  developers. 

Chapter  III  identifies  key  characteristics  of  CoPs  in  existing  literature  and 
correlates  them  to  existing  successful  communities  and  tools  that  the  military  engages 
with  or  utilizes. 

Chapter  IV  introduces  the  ideas  of  design,  design  thinking,  and  human-centered 
design.  Then  we  apply  these  methods  to  design  a  CoP  for  Navy  programmers  and 
developers. 

Chapter  V  details  the  methodology  used  to  design  the  proposed  CoP.  This  chapter 
summarizes  the  results  from  surveys,  interviews,  and  observations  conducted  during  a 
three-day  event,  HACKtheSKY.  This  chapter  also  outlines  proposed  sponsorship  for  a 
future  Navy  CoP  for  programmers  and  developers.  It  concludes  with  recommendations 
and  a  series  of  checklists  that  may  be  used  to  design  a  CoP. 

Chapter  VI  summarizes  the  research  and  analysis,  recommends  areas  for  further 
research,  and  concludes  with  recommendations  for  Tenth  Fleet  to  move  forward  with  the 
proposed  CoP  for  programmers  and  developers. 
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II.  COMMUNITIES  OF  PRACTICE 


Theorists  Jean  Lave  and  Etienne  Wenger  coined  “Communities  of  Practice”  [5]. 
Though  the  CoP  term  is  relatively  new,  the  concept  has  been  practiced  for  several 
decades  in  groups  that  have  found  that  knowledge  sharing  and  collaboration  can  provide 
valuable  benefits  for  organizations.  In  this  chapter,  Section  A  defines  a  CoP.  Section  B 
explains  why  CoPs  are  important.  Section  C  culminates  with  a  case  for  creating  a  Navy 
CoP  for  programmers  and  developers. 

A.  WHAT  IS  A  COMMUNITY  OF  PRACTICE 

Communities  of  Practice  are  groups  of  people  who  share  a  common  interest  in  a 
subject  and  collaborate  over  time  in  order  to  create  solutions  or  develop  innovations. 
There  are  various  definitions  of  Communities  of  Practice.  Communities  of  Practice  are 
“groups  of  people  who  share  a  concern  or  a  passion  for  something  they  do  and  learn  how 
to  do  it  better  as  they  interact  regularly”  [6].  They  “often  focus  on  sharing  best  practices 
and  creating  new  knowledge  to  advance  a  domain  of  professional  practice”  [7]. 
Additionally,  CoPs  are  “groups  that  form  to  share  what  they  know  and  to  learn  from  one 
another  regarding  some  aspects  of  their  work”  [8].  Knowledge  transfer  is  so  central  to  the 
idea  of  CoPs  that,  “learning  can  be  the  reason  the  community  comes  together  or  an 
incidental  outcome  of  member’s  interactions”  [6].  Therefore,  though  CoPs  may  take 
different  forms,  a  central  focus  on  the  broad  diffusion  of  expertise  is  essential  to  the 
concept. 

According  to  Wenger,  “three  characteristics  are  crucial  to  defining  a  CoP:  the 
domain,  the  community,  and  the  practice”  [6].  A  domain  has  an  identity,  typically  a 
shared  area  of  interest.  Members  of  a  CoP  are  committed  to  the  domain,  and  have  a 
shared  skill  set  that  distinguishes  them  from  the  general  populace.  The  community  is 
created  by  members  who  “engage  in  joint  activities  and  discussions,  help  each  other,  and 
share  information”  [6].  Members  within  the  community  “develop  relationships  that 
enable  them  to  share  and  leam  from  one  another”  [6].  Again,  CoPs  may  take  different 
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forms,  but  there  are  defining  characteristics  of  CoPs  that  are  common  across  different 
domains. 

Fred  Nickols  [9],  an  independent  management  consultant  who  has  authored 
numerous  CoP  guidebooks,  builds  on  Wenger’s  list  with  three  additional  characteristics  - 
joint  enterprise  through  negotiated  meaning,  mutual  engagement,  and  shared  repertoire. 
Nickols  describes  a  joint  enterprise  as  one  where  members  of  a  CoP  accomplish  a  task  on 
a  continuous  basis  with  some  kind  of  commonality  between  individuals  and  a  clear 
understanding  of  the  larger  purpose  of  the  work.  In  short,  they  have  a  unified  vision.  He 
believes  mutual  engagement  occurs  when  “the  members  of  a  CoP  interact  with  one 
another  not  just  in  the  course  of  their  work  assignments  but  to  clarify  that  work,  to  define 
how  it  is  done,  and  even  to  change  how  it  is  done”  [10].  According  to  Nickols,  shared 
repertoire  means  that,  “the  members  of  a  CoP  have  not  just  work  in  common  but  also 
methods,  tools,  techniques  and  even  languages,  stories  and  behavior  patterns”  [9].  All 
three  characteristics  and  qualities  overlap  and  are  inseparable.  Collectively,  they  work 
together  in  order  to  determine  the  practice.  Conversely,  the  practice  also  refines  the  way 
each  of  these  characteristics  is  reflected  in  the  unique  community. 

The  term  practice,  in  a  CoP,  refers  to  a  skill  set  shared  among  CoP  members.  As 
defined  by  Wenger,  “members  of  a  community  of  practice  are  practitioners.  They 
develop  a  shared  repertoire  of  resources:  experience,  stories,  tools,  ways  of  addressing 
recurring  problems — in  short  a  shared  practice”  [7].  This  shared  practice  takes  time  to 
mature  and  continues  to  develop  through  sustained  interaction.  As  defined  in  the 
Merriam-Webster  dictionary,  the  term  practice  refers  to  something  that  is  completed  on  a 
regular  basis  [11].  Practice  is  commonly  understood  as  it  refers  to  the  professional 
activity  of  doctors,  dentists  or  lawyers  and  their  practices. 

Practice  is  not  a  term  generally  used  when  referring  to  software  programmers  or 
developers;  however,  the  skills  associated  with  programming  and  software  development 
are  constantly  being  improved  upon.  They  are  developed  from  a  shared  repertoire  and 
completed  on  a  regular  basis.  Thus,  we  assert  that  programming  and  developing  software 
is  a  practice,  and  we  will  refer  to  individuals  who  engage  in  this  practice  as  programmers 
and  developers. 
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Communities  of  Practice  may  self-organize  or  be  sponsored  [12].  This  thesis 
focuses  on  sponsored  CoPs  because  of  several  structural  advantages  of  the  sponsored 
model  for  communities.  Specifically,  sponsored  CoPs  are  typically  “initiated,  chartered, 
and  supported  by  management”  which  often  results  in  monetary  support  and  resources  for 
the  community  [12].  Additionally,  sponsored  CoPs: 

1.  Enable  colleagues  to  learn  from  one  another  through  the  sharing  of  issues, 
ideas,  lessons  learned,  problems  and  their  solutions,  research  findings  and 
other  relevant  aspects  of  their  mutual  interest. 

2.  More  broadly  share  and  better  leverage  the  learning  that  occurs  in  the  CoP 
with  others. 

3.  Generate  tangible,  measurable,  value-added  benefits  to  the  business.  [12] 

Given  that  this  thesis  is  targeted  for  the  planned  design  of  a  Navy  CoP  for 
programmers  and  developers  we  naturally  focus  on  the  sponsored  model  for  a 
community. 

In  summary,  a  CoP  is  a  collaborative  team  that  communicates  and  seeks  to 
expand  knowledge  in  a  particular  practice.  A  CoP  is  defined  by  its  identity  through  a 
domain,  its  members  through  the  community,  and  the  skill  set  of  practitioners  in  the 
practice.  Sharing  of  knowledge  and  resources  eventually  evolves  into  a  shared  repertoire 
for  the  practice  of  specific  CoP-related  skills  [7].  As  practitioners,  over  time  and  through 
mutual  engagement,  members  of  the  CoP  eventually  develop  a  shared  repertoire  of 
resources.  This  shared  repertoire  may  be  used  to  improve  an  organization’s  mission. 
These  resources  could  include  tools,  experiences,  stories,  and  lessons-learned.  These 
types  of  interactions  occur  over  an  unspecified  amount  of  time  and  through  consistent 
interactions.  Finally,  the  CoP  envisioned  in  this  research  reaps  the  benefits  of  strong 
sponsorship. 

B.  WHY  ARE  COMMUNITIES  OF  PRACTICE  IMPORTANT? 

Communities  of  practice  are  important  because  they  expand  social  networks 
through  mechanisms  that  will  be  examined  in  future  sections  of  this  thesis;  however,  the 
central  assertion  of  this  work  is  that  expanded  social  networks  “help  people  organize 
around  purposeful  actions  that  deliver  tangible  results”  [13].  CoPs  are  “where  best 
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practices  and  innovation  fires  emerge  and  where  the  solutions  to  shared  problems  are  first 
identified”  [12].  It  is  the  increased  productivity,  creativity,  and  adaptability  of  the 
community  to  the  sponsor’s  challenges  that  holds  the  potential  for  a  Navy  CoP  proposed 
in  the  next  section.  Before  making  that  specific  case,  we  look  more  closely  at  the  value 
proposition  in  a  CoP’s  ability  to  build  global  networks,  stimulate  learning,  and  deliver 
quantifiable  results. 

CoPs  connect  individuals  globally  across  different  societies,  demographics,  and 
levels  of  expertise.  For  example,  a  collaboration  tool  that  enables  CoPs  to  connect 
individuals  globally,  and  provides  a  venue  to  network,  share  information,  and  enable 
dialogue  is  GitHub.  GitHub  is  an  online  code  repository  hosting  service  that  claims  to 
host  “36  million  visitors  each  day”  [14].  According  to  one  study  [15],  “the  software 
development  community  has  embraced  GitHub  as  an  essential  platform  for  managing 
their  software  projects.”  This  same  study  also  revealed  that  GitHub  “provides  a  traceable 
project  repository,  but  also  provides  a  social  meeting  place  for  all  interested  parties 
supporting  CoPs.  GitHub  is  emerging  as  a  collaborative  platform  for  education  with 
features  for  managing  and  improving — perhaps  even  transforming — the  learning 
experience”.  GitHub  supports  “open  collaboration  within  commercial  organizations  by 
centralizing  tools  and  information  and  making  them  transparent”  [16].  This  is  further 
“advocated  by  developers  acting  as  change  agents,  the  organic  adoption  of  the  tool  and  its 
corresponding  process  that  can  ensure  commonality  between  toolkits  to  utilize”  [16].  This 
collaboration  tool  has  enabled  successful  CoPs  to  connect  individuals,  organize  around  a 
certain  domain  identity,  and  provide  structures  that  promote  regular,  active  interaction 
and  the  sharing  of  knowledge. 

CoPs  are  also  important  because  they  stimulate  learning,  capture  and  diffuse 
existing  knowledge,  and  provide  a  platform  for  mentoring  and  coaching.  This  structure 
encourages  members  to  improve  within  their  practice,  because  the  CoP  introduces 
“collaborative  processes  to  organizations,  and  encourage  the  free  flow  of  ideas  and 
exchange  of  information”  [13].  Organizations  that  adopt  a  CoP  and  reap  these  benefits 
inherently  provide  their  workforce  opportunities  to  further  excel  in  their  practice. 
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An  online  CoP  that  values  learning,  mentoring,  and  coaching  is  Stack  Overflow. 
Stack  Overflow  is  an  online  question  and  answer  website  with  a  community  of 
approximately  4.7  million  programmers  [17].  Stack  Overflow  supports  these  values 
through  three  common  properties  of  social  learning  technologies  to  include:  “support  to 
learners  in  locating  the  correct  content,  allows  learners  to  connect  with  the  right  people 
(question  tags  show  what  domain  of  expertise  is  needed  and  people  with  that  expertise 
can  answer  the  question),  and  motivate/incentivize  people  to  learn”  [18].  Stack  Overflow 
provides  an  online  platform  “for  developers  to  post  their  programming  questions  and  for 
fellow  developers  to  provide  answers”  [19].  The  website  allows  anyone  interested  in 
programming  to  join,  ask  questions,  and  receive  timely  feedback.  As  the  questioner,  they 
“can  add  a  few  tags  to  questions  to  help  others  (e.g.  potential  responders)  better 
understand  the  question  as  well  as  select  one  answer  as  the  most  helpful  one,  and  then 
site  members  can  vote  on  questions  and  answers”  [19].  The  quality  of  questions  and 
answers  is  determined  by  its  dedicated  community  of  users  who  vote  on  and  improve 
posts  through  constructive  feedback  and  have  the  ability  to  reject  questions  not  pertaining 
to  the  identified  focus  [19].  Furthermore,  Stack  Overflow  places  significant  importance 
on  its  members  and  encourage  them  through  reputation  points  to  take  ownership  of  the 
website  and  community  at  large.  Through  active  member  participation,  the  free  flow  of 
ideas,  constructive  feedback,  and  collaboration,  Stack  Overflow  members  improve  the 
practices  of  each  other  through  shared  learning,  mentoring,  and  coaching. 

Communities  of  practice  are  also  important  because  the  community  collaborates 
to  produce  useful  products  [13].  A  successful  CoP  that  exemplifies  this  value  is 
TopCoder,  a  company  that  administers  and  hosts  technical  competitions  in  computer 
programming.  With  1,020,945  active  members  (as  of  May  3,  2016),  it  is  one  of  the 
largest  online  technical  communities  in  the  world  [20].  TopCoder  focuses  on  competition 
and  community.  TopCoder  is  a  crowdsourcing  platform  used  by  enterprises  to  “deliver 
their  software  developing  tasks”  to  be  competed  for  by  TopCoder  members  [21].  Though 
TopCoder  is  competitive  in  spirit,  one  study  observed  that  their  competitions  are  “never 
disrespectful  or  nasty,  instead  people  like  to  help  each  other”  [22].  TopCoder  forums  are 
“the  main  source  for  collaboration,  where  less  experience  community  members  ask  for 
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assistance  on  certain  problems  and  receive  instant  feedback  from  more  experienced 
competitors”  [22].  These  forums  provide  members  “opportunities  to  network 
professionally  and  socially”  [22].  Aside  from  collaboration,  the  competition  mechanism 
has  “ensured  high-quality  solutions”  for  each  phase  of  competition  [21].  Sponsors  of 
TopCoder  shape  competitions  in  order  to  compete  and  deliver  “cost  effective  and  time 
efficient”  products  [22].  This  exemplifies  how  a  community  can  collaborate  together  to 
produce  tangible  results. 

In  summary,  CoPs  are  important  because  of  the  tangible  results  they  yield,  but 
these  measurable  outcomes  are  the  product  of  less  quantifiable  values.  One  of  the  most 
important  intangible  values  of  a  CoP  is  the  expanded  social  network,  which  fosters 
learning  through  knowledge  sharing  and  provides  mentorship  and  coaching  opportunities. 
Additionally,  CoPs  “capture  and  diffuse  existing  knowledge,  while  introducing 
collaborative  processes  to  help  people  organize  and  generate  new  ideas”  [13].  Therefore, 
the  Navy  may  benefit  from  a  CoP  for  programmers  and  developers. 

C.  A  CASE  FOR  A  NAVY  COMMUNITY  OF  PRACTICE  FOR 

PROGRAMMERS  AND  DEVEEOPERS 

The  Navy’s  formal  and  informal  cyber  workforces  stand  to  benefit  from  a  CoP 
sponsored  by  the  formal  cyber  workforce.  A  CoP  would  provide  tangible  benefits  for  the 
formal  cyber  workforce  to  include  increased  effectiveness  and  mentorship  in  the 
execution  of  cyber  operations.  Tangible  benefits  to  the  informal  cyber  workforce  include 
expanded  professional  networking  and  coaching  that  could  result  in  increased 
cybersecurity  for  the  creation,  sustainment,  and  operation  of  computer-dependent  systems 
outside  of  those  dedicated  to  the  execution  of  formal  cyber  operations.  A  Navy  CoP  for 
programmers  and  developers  could  provide  these  benefits  while  simultaneously 
providing  an  operational  mechanism  to  deliver  outcomes  articulated  by  the  Chief  of 
Naval  Operations  and  the  Secretary  of  Defense. 

In  order  to  articulate  the  benefits  of  a  Navy  CoP  to  the  formal  cyber  workforce  it 
is  important  to  first  understand  a  little  more  about  the  structure  of  the  officers  engaged  in 
cyber  tool  development.  Cyber  Warfare  Engineers  (CWEs)  comprise  the  active  duty 
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naval  officer  component  of  cyber  tool  development  within  the  Cryptologic  Warfare 
Community  [23].  A  structural  challenge  identified  in  anecdotal  conversations  within  this 
small  cadre  tasked  with  programming  in  support  of  Fleet  Cyber  Command  operations  and 
missions  is  the  lack  of  coaching  and  mentorship  provided  by  experienced  software 
developers  to  the  junior  officer  CWEs.  After  their  five-year  obligation  of  service  CWEs 
matriculate  out  of  the  program  and  must  transfer  into  different  communities  within  the 
Navy  or  resign  their  commission  and  move  on  to  a  civilian  career,  leaving  the  next 
generation  of  CWEs  without  experienced  mentors.  This  matriculation  and  transferring  of 
CWEs  does  not  all  happen  at  once,  they  are  continuously  rotating  in  and  out  of  the  ranks 
and  workforce.  The  formal  cyber  workforce  stands  to  take  advantage  of  this  global 
network  of  former  CWEs  to  address  the  lack  of  mentoring  and  coaching  they  currently 
experience.  The  structure  and  design  of  a  CoP  would  allow  for  both  current  and  past 
CWEs  to  socially  network,  share  knowledge,  and  coach  one  another.  A  CoP  would  offer 
global  accessibility  to  all  CWEs,  reducing  loss  of  knowledge  and  experience  and  could 
alleviate  some  of  the  lack  of  coaching  and  mentorship  challenges  experienced  among 
CWEs  due  to  their  community  structure  and  matriculation  rate. 

Another  structural  challenge  identified  in  anecdotal  conversations  within  the 
Cryptologic  Warfare  community  reveal  that  some  Cryptologic  Warfare  officers  are 
inclined  to  focus  on  cyberspace  operations,  but  must  rotate  between  other  areas  in  order 
to  compete  for  promotion.  For  example,  after  many  officers  complete  a  tour  executing 
cyberspace  operations  they  are  then  encouraged  to  diversify  their  career  with  a  tour  in 
another  field  such  as  signals  intelligence.  This  creates  officers  who  are  generalists  across 
a  broad  field  of  naval  capabilities  but  diffuses  the  expertise  of  officers  who  are  more 
inclined  to  focus  on  the  practice  of  programming  and  developing  as  related  to  cyberspace 
operations.  Creation  of  a  Navy  CoP  for  programmers  and  developers  would  allow 
experienced  Cryptologic  Warfare  officers  with  an  interest  in  cyberspace  operations  a  way 
to  stay  connected  and  close  to  the  community  while  they  serve  in  billets  aligned  to 
signals  intelligence  or  other  responsibilities. 

Next,  a  Navy  CoP  could  bridge  the  gap  between  the  formal  and  informal  cyber 
workforces  caused  by  different  levels  of  experience  and  classification  barriers.  Bridging 
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this  gap  could  provide  the  tangible  benefit  of  increased  cybersecurity  for  Navy 
capabilities  outside  of  the  Tenth  Fleet  mandate.  The  large  informal  cyber  workforce  that 
includes  requirements  definition  at  the  Office  of  the  Chief  of  Naval  Operations  level, 
program  oversight  from  the  many  Program  Executive  Offices,  and  systems  command 
programmers  and  developers  is  challenged  with  the  responsibility  of  generating  platforms 
that  are  exposed  to  adversary  cyberspace  operations,  but  the  system  developers  often  lack 
knowledge  of  enemy  capabilities.  In  addition  to  this  shortfall  in  knowledge,  the 
classification  levels  of  systems,  tools,  and  capabilities  often  discourages  cross¬ 
collaboration  among  the  formal  and  informal  cyber  workforce  resulting  in  systems 
designed  without  full  visibility  of  the  operating  environment  in  which  they  will  be 
expected  to  fight  and  win.  By  comparison,  the  formal  cyber  workforce  is  much  smaller 
than  the  large  community  of  Navy  programmers  and  developers  in  the  informal 
system  [24].  However,  the  small  formal  workforce  is  much  more  familiar  with  adversary 
cyber  warfare  capabilities  and  its  members  are  cleared  and  granted  access  to  the  required 
classified  information  needed  to  conduct  their  mission  unlike  their  counterparts.  A  CoP 
would  unite  the  formal  and  informal  workforces  through  collaboration,  and  enable 
dialogue  resulting  in  a  repertoire  of  shared  knowledge,  best  practices,  and  techniques. 
The  desired  output  from  this  increased  collaboration  and  knowledge  diffusion  could  be 
increased  cybersecurity  of  computer  dependent  systems  designed  and  fielded  by  the 
informal  workforce. 

Additionally,  we  assume  that  a  CoP  would  address  needs  of  both  the  formal  and 
informal  cyber  workforces  while  simultaneously  supporting  the  Chief  of  Naval 
Operations  (CNO)  desired  goal  of  high  velocity  learning.  The  CNO  defines  high  velocity 
learning  as  “a  Naval  force  that  produces  leaders  and  teams  who  learn  and  adapt  to 
achieve  maximum  possible  performance,  and  who  achieve  and  maintain  high  standards  to 
be  ready  for  decisive  operations  and  combat”  [25].  This  definition,  taken  from  the  CNO’s 
A  Design  for  Maintaining  Maritime  Superiority,  reflects  his  highest  priority  guidance  to 
the  Navy.  An  expansion  of  the  Navy’s  formal  and  informal  workforces  to  include 
collaboration  with  industry  could  enhance  high  velocity  learning  because  of  the  shared 
knowledge  and  experience  among  workforces. 
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As  highlighted  in  the  Secretary  of  Defense’s  (SECDEF’s)  recent  DOD  cyber 
strategy  document,  “to  succeed  in  its  mission  the  DOD  must  operate  in  partnership  with 
other  Departments  and  Agencies,  international  allies  and  partners,  state  and  local 
governments,  and,  most  importantly,  the  private  sector”  [26].  (Note:  we  use  the  term 
industry  to  identify  the  private  sector  of  programmers  and  developers  referred  to  in  the 
SECDEF  memo.)  We  believe  a  Navy  CoP  that  includes  industry  participation  would 
operationalize  the  partnership  that  SECDEF  is  encouraging,  and  expose  the  Navy  to 
industry’s  best  practices  in  software  development  and  programming.  Without  a 
mechanism  to  bring  about  the  aforementioned  partnerships  the  formal  and  informal 
workforces  may  not  be  aware  of  the  latest  cybersecurity  technologies.  Exposure  to 
industry  would  allow  both  workforces  to  better  understand  cutting  edge  tactics, 
techniques,  tools,  and  procedures  that  could  be  used  to  improve  and  quicken  the  Navy’s 
iteration  cycle  of  tool  and  capability  development. 

Another  important  reason  for  including  industry  in  a  proposed  Navy  CoP  is  to 
bridge  the  gap  between  commercial  cybersecurity  and  military  cyber  warfare.  As  a 
military  force  the  Navy  conducts  cyberspace  operations  under  a  different  set  of 
permissions  and  laws  than  the  commercial  sector.  U.S.  Code  Title  10  and  Title  50 
authorities  govern  Navy  conduct  for  cyber  warfare,  which  is  fundamentally  different  than 
commercial  cybersecurity  laws  enacted  in  the  Cyber  Security  Act  of  2015,  Cyber 
Security  Enhancement  Act,  Electronic  Communications  Privacy  Act,  Digital  Millennium 
Copyright  Act,  or  the  Computer  Fraud  and  Abuse  Act  [27]. 

As  previously  mentioned,  military  and  industry  must  abide  by  different  laws  and 
restrictions  when  conducing  cyberspace  operations.  A  CoP  would  provide  a  forum  for 
industry  to  learn  about  the  nuances  of  Navy  requirements  that  are  different  from  their 
other  customers.  Potential  challenges  stemming  from  acquisition  or  export  control 
policies  and  regulations  for  example,  could  be  addressed  as  the  community  evolves.  If 
industry  were  an  active  participant  in  the  CoP  then  knowledge  transfer  to  and  from  the 
Navy  would  occur.  Industry  would  benefit  from  a  more  comprehensive  understanding  of 
a  cyber  warfare  paradigm  the  Navy  operates  under,  allowing  industry  to  further  innovate 
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and  improve  their  tools  and  capabilities  to  assist  in  improving  Navy  processes  and 
capabilities. 

In  summary,  this  thesis  assumes  that  a  CoP  provides  bi-directional  tangible 
benefits  to  the  formal  and  informal  cyber  workforces  as  well  as  industry.  The  formal 
cyber  workforce  benefits  from  the  global  access  to  former  CWEs  who  would  provide  the 
needed  support  and  mentorship  to  future  CWEs.  The  formal  cyber  workforce  also  stands 
to  benefit  from  increased  productivity  among  its  Cryptologic  Warfare  officers  who  are 
inclined  to  focus  on  cyberspace  operations  even  while  assigned  to  billets  in  other  mission 
areas.  The  informal  cyber  workforce  would  benefit  from  knowledge  required  to  increase 
cybersecurity  of  computer-dependent  systems  outside  of  Fleet  Cyber  Command’s  span  of 
control.  Industry  would  benefit  from  exposure  to  the  Navy’s  cybersecurity  needs  and 
requirements,  and  the  Navy  would  benefit  from  exposure  to  industry’s  best  practices, 
tools,  and  techniques.  All  of  these  benefits  are  due  to  the  structural  way  that  CoPs  bring 
individuals  together.  Expanded  social  networks  allow  individuals  to  apply  knowledge  in 
ways  that  may  result  in  “new  solutions  to  old  and  new  problems”  [9].  Therefore,  due  to 
the  potential  tangible  benefits  to  be  gained,  it  is  in  the  best  interest  of  the  Navy  to  design 
and  sponsor  a  Navy  CoP  for  programmer  and  developers  with  the  characteristics 
identified  in  the  following  chapter. 
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III.  KEY  CHARACTERISTICS  OF  SUCCESSFUL  COMMUNITIES 

OF  PRACTICE 


All  CoPs  are  unique;  however,  successful  CoPs  often  have  common  traits. 
Chapter  III  will  focus  on  three  shared  key  characteristics  of  successful  CoPs  to  include: 
social  interactions,  structure,  and  lifecycle.  This  chapter  also  examines  existing 
communities  that  have  been  successful  by  exhibiting  the  aforementioned  characteristics. 
These  existing  communities  and  characteristics  should  be  used  as  a  point  of  consideration 
when  designing  a  Navy  CoP  for  programmers  and  developers. 

A.  SOCIAL  INTERACTIONS 

Successful  CoPs  promote  social  interactions  through  collaboration,  discussion, 
and  knowledge  sharing.  These  interactions  may  occur  in  various  environments  to  include: 
“face-to-face,  video/audio  teleconference,  telephone,  electronic  mail,  and  website  access 
devices”  [28].  Furthermore,  these  interactions  should  generate  energy  that  encourages 
active  engagement  and  participation.  There  has  been  limited  recent  research  conducted  on 
the  value  of  periodic  face-to-face  meetings  compared  against  virtual  communities. 
Despite  this  shortfall  in  literature,  this  thesis  will  explore  two  common  forms  of  social 
interactions,  face-to-face  and  virtual  technology,  as  seen  in  successful  communities,  and 
how  these  environments  contribute  to  the  success  of  communities. 

Face-to-face  interactions  involve  two-way  dialogue  where  immediate  feedback 
can  be  received,  allowing  individuals  to  better  express  and  gauge  how  ideas  are  being 
perceived.  A  2013  study  by  the  Harvard  Business  Review  revealed,  “87%  of 
professionals  believe  face-to-face  meetings  are  essential  for  completing  a  business 
transaction,  and  95%  said  they  are  key  to  successful,  long-lasting  business  relationships” 
[29].  Face-to-face  interactions  are  intended  to  bring  individuals  together  with  the  intent  to 
share  and  express  ideas  and  information.  Face-to-face  interactions  allow  a  diverse  group 
of  individuals  to  interact  in  an  environment  where  their  message  is  heard  with  a 
decreased  chance  of  miscommunication  or  interpretation. 
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The  Athena  Project  is  a  community  that  thrives  on  frequent  face-to-face  social 
interactions  in  various  geographical  areas.  The  Athena  Project  was  an  “initiative  founded 
in  the  Navy  Surface  Community  aboard  USS  BENEFOLD  (DDG  65)  in  early  2013”  [30]. 
The  Athena  Project  focuses  on  “harnessing  deckplate  innovations  to  create  a  cadre  of 
forward-thinking,  creatively  confident  Sailors  of  the  Fleet  of  tomorrow”  [30].  Utilizing 
an  open  forum  platform.  Sailors  can  “pitch  innovative  ideas  to  improve  their  command  or 
the  Navy”  [30].  The  Athena  Project  aims  to  host  quarterly  events  for  San  Diego  and 
Norfolk  naval  bases  as  well  as  the  Pacific  Northwest.  These  events  provide  Sailors  a 
venue  to  share  and  develop  their  ideas  to  improve  their  unit  or  service,  and  ultimately 
affect  change.  The  successes  of  Athena  Project  events  are  attributed  to  face-to-face  social 
interactions  that  bring  together  individuals  from  all  communities  within  the  Navy,  and 
provide  them  a  venue  to  share  ideas  and  concerns  in  order  to  improve  processes  and 
policies  within  the  Navy.  Success  stories  shared  on  the  Athena  Project  website  include 
projects  that  have  been  “published  as  tactics,  techniques,  and  procedures  that  have  also 
been  adopted  throughout  the  fleet”  [30].  As  demonstrated  by  the  Athena  Project,  face-to- 
face  interactions  may  occur  in  small  exclusive  settings,  or  large  international  events 
discussed  next. 

Cybersecurity  conferences  provide  energy  and  engagement  that  take  advantage  of 
the  benefits  gained  from  social  interactions.  Two  well-known  annual  cybersecurity 
conferences  that  showcase  hacker  practices  to  a  range  of  participants  are  DefCon  and 
Black  Hat.  DefCon  is  hosted  annually  in  Las  Vegas,  and  is  one  of  the  oldest  open 
cybersecurity  conferences  in  existence  [31].  DefCon  “attracts  top  level  cybersecurity 
researchers  and  hackers  interested  in  software,  computer  architecture,  hardware 
modification,  and  code-scripting”  [31].  Black  Hat  was  developed  from  DefCon  20  years 
ago  and  attracts  8,000  attendees  each  year  [32].  Cybersecurity  conferences  such  as  these 
provide  an  opportunity  for  professionals  to  learn  about  developing  technology,  receive 
valuable  insights  from  cyber  leading  experts,  and  network  with  other  professionals  in  the 
cyber  workforce.  Face-to-face  interactions  allow  a  diverse  group  of  individuals  from  all 
walks  of  life  to  interact  and  share  information  with  the  expectation  of  immediate 
feedback. 
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Given  consistent  advancements  in  technology  and  newer  generations  learning 
how  to  operate  computers  early  in  life,  virtual  interactions  often  appeal  more  to  the 
“millennial”  generation  [33].  A  research  study  [34]  on  virtual  teams  conducted  in  2015 
examined  the  past  ten  years  of  literature  and  studies  on  virtual  teams.  One  benefit  of 
virtual  teams  was  outreach  to  the  millennial  generation.  The  study  stated  that  the 
millennial  generation’s  exposure  to  virtual  methods  at  a  young  age  has  increased  their 
comfort  and  acceptance  of  virtual  technology  as  a  means  to  socially  interact. 
Additionally,  the  millennial  generation  “has  the  ability  to  effectively  utilize  broadly 
networked  digital  communication  technologies  to  quickly  and  seamlessly  accomplish  a 
wide  variety  of  tasks”  [35].  Furthermore,  working  together  virtually  aligns  with  the 
“values  and  expectancies  of  younger  employees”  who  “place  a  greater  value  on  work-life 
balance”  [34].  In  order  for  a  community  to  be  successful  through  the  use  of  virtual 
interactions,  “members  of  an  organization  need  to  be  comfortable  with  participating  in  a 
computer-mediated,  internet-based  community  of  practice,  which  involves  very  little 
face-to-face  communication”  [36].  The  key  to  virtual  interactions  is  a  virtual  platform 
that  is  simple  to  leam  and  operate  for  all  individuals. 

Web-based  platforms  are  ideal  for  virtual  interactions  because  of  their  ease  of 
operations  and  ability  to  connect  a  global  audience.  All  Partners  Access  Network 
(APAN)  is  a  “U.S.  DOD  social  networking  website  that  has  been  used  for  over  a  decade 
primarily  for  information  sharing  and  global  collaboration”  [37].  APAN’s  website  [37] 
hosts  a  network  of  communities  that  allow  users  to  post  multimedia,  blogs,  wikis,  forums, 
documents,  and  a  variety  of  information  to  a  communities’  specific  page.  Their  website 
provides  organizations  and  individuals  with  unclassified  tools  to  communicate 
worldwide.  Collaboration  tools  offered  on  APAN’s  webpage  can  be  utilized  standalone 
or  combined  in  order  to  design  and  develop  a  specific  community  space.  Furthermore, 
APAN’s  website  provides  a  virtual  place  for  individuals  worldwide  to  connect  and 
collaborate.  Virtual  interactions  are  beneficial  to  CoPs  because  of  their  low  cost  to 
operate,  decrease  in  travel  time,  and  ability  to  rapidly  make  decisions  around  the 
globe  [38].  More  often  than  not,  the  audience  of  a  CoP  will  determine  the  type  of  social 
interaction  that  is  most  appropriate  and  more  likely  to  contribute  to  the  CoP’s  success. 
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In  conclusion,  social  interactions  support  the  importance  of  CoPs  by  connecting 
practitioners,  encouraging  collaboration,  and  enabling  knowledge  sharing.  Social 
interactions  can  be  conducted  face-to-face  or  virtually.  The  type  of  social  interaction 
selected  by  a  CoP  is  often  determined  by  the  intended  audience,  and  takes  into 
consideration  their  experience  and  needs.  Both  forms  of  social  interactions  have 
advantages  and  provide  benefits  to  a  CoP,  and  both  will  be  necessary  to  sustain  the 
proposed  Navy  CoP  for  programmers  and  developers  because  of  the  diverse  experience, 
demographics,  and  the  geographic  mobility  of  its  members. 

B.  STRUCTURE 

Successful  CoPs  operate  in  various  ways,  but  they  share  four  common  structural 
elements:  1)  sponsors,  champions,  and  stakeholders,  2)  mission,  vision,  and  purpose,  3)  a 
membership  framework,  and  4)  measures  of  success.  This  section  explores  these 
structures  and  includes  examples  of  each  in  successful  communities. 

1.  Sponsors,  Champions,  and  Stakeholders 

Successful  CoPs  rely  on  sponsors,  champions,  and  stakeholders  to  promote,  grow, 
and  sustain  their  community.  It  is  important  to  have  a  committed  sponsor  “who  is  able  to 
envision  the  benefits  of  a  community  of  practice  over  time,  and  has  a  sense  of  how  the 
community  can  interact  across  sectors”  [39].  Furthermore,  committed  sponsors  are 
“invaluable  to  the  resourcing  and  sustainability  of  a  community”  [39]. 

Sponsors  are  “the  bridge  between  a  CoP  and  the  rest  of  the  formal  organization, 
particularly  the  authority  hierarchy”  [40].  The  sponsors’  responsibilities  include: 
“articulate  CoP  mission,  manage  official  relationships,  remove  barriers  and  obstacles 
preventing  CoP  productivity,  run  interference  as  necessary,  and  provide  funding  as 
needed”  [40].  A  CoP  may  have  one  or  more  sponsors. 

A  CoP  champion  is  “the  chief  organizer  of  events”  [40].  The  champion’s 
responsibilities  include:  “ensure  and  articulate  valid  CoP  purpose(s),  generate  CoP 
interest,  organize  face-to-face  events,  stimulate  energy  and  enthusiasm,  promote 
participation  and  contributions  of  CoP,  obtain  official  support  as  needed,  communicate 
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contributions  of  CoP  to  formal  organization  or  seniors,  and  communicate  company 
support  to  CoP  members”  [40].  The  champion  is  the  glue  that  brings  together  the 
members  of  a  CoP. 

Stakeholders  of  a  CoP  are  people  or  organizations  internal  or  external  to  the  CoP 
that  have  an  interest  in  the  CoP’s  mission,  as  well  as  influence  over  the  efforts  and 
outcomes  of  the  CoP  [41].  The  stakeholder’s  responsibilities  may  include:  “reviewing, 
approving,  and  funding  of  deliverable  project  items”  [42].  Stakeholders  may  strengthen 
the  organization’s  relationships  with  other  stakeholders,  because  of  common  shared 
interests  in  CoP  mission  accomplishment.  Stakeholders  provide  a  CoP  with  credibility 
and  legitimacy,  often  due  to  the  stakeholder’s  rank,  status,  or  reputation.  However, 
stakeholders  can  also  act  as  a  barrier  to  the  evolution  of  a  CoP  [41]. 

Stakeholders  often  have  the  most  influence  over  the  success  of  CoP  because  of 
their  vested  interest  in  the  CoP’s  outcomes  and  products.  The  Joint  Interagency  Field 
Experimentation  (JIFX)  program  is  an  excellent  example  of  how  important  stakeholders 
are  to  the  success  of  a  community.  JIFX  is  a  Naval  Postgraduate  School  (NPS)  program 
that  hosts  quarterly  events  at  Camp  Roberts,  California  Army  National  Guard  Training 
camp  in  order  to  “provide  a  field  experimentation  resource  for  its  stakeholders — the 
Unified  Combatant  Commands  and  other  federal  agencies  to  inform  the  requirements 
generated  by  these  entities”  [43].  JIFX  is  “sponsored  by  the  Office  of  the  Secretary  of 
Defense  and  the  Department  of  Homeland  Security”  [43].  JIFX’s  primary  stakeholders 
are  the  higher  echelon  commands,  and  secondary  stakeholders  include  all  other  DOD 
organizations,  interagency  and  federal  organizations,  and  state  and  local  government 
organizations  [43].  The  stakeholders  define,  delineate,  and  eliminate  requirements  for 
JIFX  events.  JIFX’s  decade  of  successes  may  be  attributed  to  their  meaningful 
stakeholders,  as  well  as  their  sponsors  that  motivate  individuals  to  join  the  community, 
solve  issues,  and  produce  tangible  results  for  current  military  projects  and  missions.  The 
legitimacy  and  credibility  gained  from  stakeholders’  reputations  within  the  government 
may  positively  influence  participation  and  outcomes  of  the  CoP. 

In  conclusion,  sponsors,  champions,  and  stakeholders  provide  invaluable 

resources  and  support  to  a  CoP.  Their  legitimacy  and  credibility  often  garner  active 
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participation  and  influence  events  that  produce  tangible  products.  They  promote  the  CoP 
and  fund  events.  Their  support  and  active  involvement  is  important  to  the  sustainment  of 
a  CoP. 


2.  Mission,  Vision,  and  Purpose 

Structural  elements  common  to  successful  CoPs  are  clear  and  concise  mission  and 
vision  statements  and  a  well-defined  purpose.  Mission  statements  should  clarify  the 
organization’s  measurable  goals  and  objectives.  The  vision  statement  further  explains  the 
CoP’s  values.  These  statements  should  be  short  and  concise,  but  provide  enough 
information  that  a  potential  member  can  clearly  envision  how  they  may  benefit  or 
contribute  as  a  future  CoP  member.  If  appropriate,  mission  and  vision  statements  should 
be  aligned  with  those  of  the  CoP’s  sponsoring  organizations  [13].  These  statements 
should  be  easy  to  access  in  order  for  all  interested  individuals  to  understand  a  CoP’s 
purpose. 

Successful  communities  make  their  mission  and  vision  statements  accessible  for 
the  public  to  view.  The  Military  Cyber  Professionals  Association  (MCPA)  provides  clear, 
concise,  and  well-defined  mission,  values,  and  goals  statements  on  their  website  [44].  In 
2013,  Army  Major  Joseph  Billingsley,  while  an  NPS  student,  identified  a  “need  for  a 
cyber  association  in  order  to  merge  the  myriad  of  professionals  in  the  cyber  operations 
field”  [45].  According  to  the  MCPA  website  [44],  MCPA’s  mission  is  “dedicated  to 
developing  the  American  military  cyber  profession  and  investing  in  our  nation’s  future 
through  Science,  Technology,  Engineering,  and  Math  education”  [44].  Their  vision  is  to 
establish  “an  American  military  cyber  profession  that  is  accomplishing  what  our  nation 
needs,  expects,  and  deserves”  [44].  MCPA  only  recruits  members  who  believe  in  their 
mission,  vision,  and  support  improving  cyber  interests,  thereby  putting  extra  emphasis  on 
the  importance  of  clear  and  concise  mission  and  vision  statements  accessible  to  all 
interested  individuals.  Public  access  to  CoP  mission  and  vision  statements  inform 
interested  participants  what  the  goals  of  a  CoP’s  and  goals  purpose. 

Successful  CoPs  have  well-defined  purposes  that  highlight  the  value  of  active 
participation  in  the  CoP.  Two  common  purposes  found  in  successful  CoPs  are  learning 
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through  shared  knowledge  and  collaboration,  and  the  act  of  taking  purposeful  action  in 
order  to  successfully  execute  tasks  and  projects.  A  CoP  evolves  when  members  learn 
from  each  other  and  develop  a  shared  practice.  A  “balance  between  the  production  of 
tools  and  deep  learning  experiences  for  CoP  members”  is  key  to  successful  practice 
development  [13].  Purpose  statements  often  focus  on  benefits  gained  as  members  of  a 
community.  CoP  purposes  are  typically  member-focused  in  contrast  to  mission  and  vision 
statements,  which  are  sponsor  outcome  focused. 

In  conclusion,  successful  CoPs  use  mission,  vision,  and  purpose  statements  to 
harmonize  their  internal  and  external  messaging  about  the  community.  Mission 
statements  outline  the  CoP’s  objectives,  and  goals.  Vision  statements  further  outline  the 
CoP’s  values.  The  purpose(s)  of  a  CoP  highlight  to  members  the  benefits  gained  from 
shared  knowledge,  collaboration,  and  actions  taken  that  result  in  tangible  results. 
Mission,  vision,  and  purpose  should  be  displayed  for  the  public  to  see  in  order  to  seek 
and  inspire  individuals  to  become  an  active  member  of  the  CoP. 

3.  Membership  Framework 

A  third  structural  element  of  successful  CoPs  is  clear  membership  frameworks. 
The  desirable  characteristics  of  a  membership  framework  that  we  want  to  emulate  are 
inclusive  membership  and  commitment.  Inclusive  membership  benefits  include  the 
expansion  of  social  boundaries  such  that  cross-collaboration  results  in  a  diverse  spectrum 
of  solutions  to  a  problem.  Committed  member  benefits  include  a  sense  of  identity  shared 
among  CoP  members,  which  contributes  to  their  commitment  to  each  other  and  the 
success,  and  longevity  of  the  CoP. 

Inclusive  membership  enables  recruitment  from  a  more  diverse  group  of 
individuals  with  different  mindsets  and  opinions,  which  can  bolster  the  productivity  and 
diversity  in  ideas  gained  through  CoP  events.  Massive  Mutiplayer  Online  Wargame 
Leveraging  the  Internet  (MMOWGLI)  is  a  platform  used  by  the  Navy  to  crowdsource 
ideas  and  solutions  to  some  of  the  Navy’s  current  problems  [46].  MMOWGLI  has 
previously  been  “used  by  the  Office  of  Naval  Research  and  other  U.S.  government 
agencies  to  perform  online  “wargames”  to  study  various  problems  and  hypothetical 
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scenarios”  [47].  MMOWGLI  harnesses  the  potential  of  large,  diverse  groups  for  thinking 
and  responding  to  emergent  challenges  and  opportunities  [48].  MMOWGLI  provides  a 
forum  for  individuals  to  share  knowledge  and  socially  interact,  which  could  add  value  to 
rapidly  developing  ideas  for  how  to  achieve  a  CoP’s  desired  outcomes  and  impacts. 
MMOWGLI’ s  inclusive  membership  invites  all  interested  individuals,  regardless  of 
association  or  geographic  location,  to  participate  in  wargames  and  share  their  ideas  for 
how  to  combat  Navy  problems.  Inclusive  membership  is  ideal  for  a  successful  Navy  CoP 
for  programmers  and  developers,  because  of  the  potential  increased  number  of 
participants  and  innovative  thinking  that  may  emerge  from  encouraging  dialogue  across  a 
worldwide  spectrum  of  individuals. 

The  commitment  of  CoP  members  can  be  affected  by  the  shared  identify  of  its 
members,  often  resulting  in  longer  lasting  membership  commitment.  The  commitment  of 
CoP  members  directly  affects  the  longevity  of  a  CoP.  The  United  States  Navy’s  Chief’s 
Mess,  while  an  exclusive  CoP,  is  a  well-established  CoP  within  the  Navy  with  a  long¬ 
standing  history,  tradition,  and  firm  commitment  to  and  from  each  member.  They  extend 
its  membership  to  all  active  and  retired  indoctrinated  Chief  Petty  Officers.  The  title  of 
“Chief’  is  a  very  distinctive  and  personal  form  of  identity.  This  unique  identity  further 
extends  the  understanding  of  deep  trusting  relationships  among  members  of  The  Chiefs 
Mess.  The  Chiefs  within  this  organization  maintain  a  lifetime  commitment  to  each  other. 
They  build  their  identity  through  shared  information,  trust,  activities,  a  commitment  to 
personal  and  professional  growth  of  the  Sailors  they  are  entrusted  with  leading,  and  the 
successful  completion  of  the  Navy’s  many  missions.  This  identity  also  adds  to  the  sense 
of  cohesion  that  is  found  within  The  Chief’s  Mess.  This  fostered  sense  of  identity  forms 
one  of  the  largest  CoPs  in  any  of  the  military  branches.  The  focus  of  identity  adds  a 
human  dimension  to  the  idea  of  practice  and  identifies  the  level  of  commitment  to  others 
who  share  that  same  practice.  CoPs  require  significant  and  sustained  levels  of 
participation  from  theirs  members  in  order  to  persist. 

In  conclusion,  the  envisioned  Navy  CoP  for  programmers  and  developers  should 
reflect  the  inclusiveness  of  participants  as  exhibited  MMOWGLI,  and  the  commitment  of 
its  members  as  exhibited  in  the  Chief’s  Mess.  While  MMOWGLI  and  the  Chief’s  Mess 
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exhibit  the  right  characteristics  to  emulate  when  creating  the  proposed  Navy  CoP  for 
programmers  and  developers,  there  is  potential  for  unforeseen  challenges  to  a  design  that 
includes  inclusive  membership  that  fosters  membership  commitment.  These  challenges 
may  be  overcome  in  proper  design  and  attention  given  to  the  types  of  interactions  that 
encourage  inclusiveness  and  commitment. 

4.  Measures  of  Success 

A  fourth  structural  element  common  to  successful  CoPs  is  the  establishment  of 
metrics  to  quantify  success.  Success  is  subjective  and  may  be  determined  by  anything 
including  the  number  of  active  members,  events,  successful  outcomes,  or  the  CoP’s 
lifespan.  CoP  sponsors,  champions,  stakeholders,  and  members  all  provide  multiple 
perspectives  on  what  constitutes  success.  Therefore  it  is  important  to  clearly 
communicate  how  success  is  envisioned.  Continuous  evaluations  of  success  should  be 
conducted  in  order  to  analyze  and  measure  the  rate  of  success  and  adjustments  made  for 
CoP  improvement  and  sustainment. 

Established  metrics  used  to  evaluate  a  CoP’s  success  are  imperative  to  the 
sustainment  and  further  support  of  a  CoP  in  order  to  manage  expectations  and  make 
adjustments  as  necessary.  CompanyCommand,  an  online  community  that  has  been 
connecting  Army  company  commanders  for  over  13  years,  exemplifies  the  importance 
and  benefits  of  established  and  defined  metrics  for  how  to  measure  success  [49]. 
CompanyCommand’ s  initial  success  was  evaluated  by  the  use  of  metrics  to  include 
“quantitative  measures  of  unique/repeat  visitors,  number  of  downloads,  subjects  searched 
for  and  found,  submission  rates,  and  time  saved  in  wheel  re-invention”  [50].  In  2002,  the 
CompanyCommand.com  site  served  “352,000  unique  visitors  who  downloaded  136 
gigabytes  of  information,  logged  16  million  hits,  and  viewed  2.7  million  pages”  [50]. 
These  quantitative  measures  made  by  CC  champions  and  stakeholders  eventually 
garnered  further  support  from  additional  sponsors.  In  2003,  as  operations  in  Iraq  gained 
momentum  and  frequency,  more  Army  Officers  began  utilizing  the  website  to  assist  them 
while  deployed.  The  Army  eventually  recognized  the  value  of  this  community  and 
“pledged  to  support  the  further  development  and  growth  of  their  CoP  as  well  as  other 
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supporting  CoPs  at  other  levels  of  leadership  (i.e.  PlatoonLeader.net)”  [50].  Established 
and  pre-determined  metrics  and  methods  to  measure  rate  of  success  may  generate  further 
CoP  support  and  sponsorship. 

How  one  CoP  measures  success  may  not  be  what  works  best  for  another  CoP,  but 
many  successful  CoPs  have  used  metrics  to  generate  and  sustain  sponsorship.  Sponsors 
may  provide  valuable  inputs  for  how  to  measure  success  or  envision  what  it  looks  like. 
The  continuous  evaluation  of  metrics  used  to  measure  success  of  a  CoP  is  necessary  in 
order  to  adjust  for  changes  as  needed.  Two  of  the  characteristics  shared  by  successful 
CoPs,  social  interactions  and  structures,  all  contribute  to  the  third  shared  characteristic, 
lifecycle. 

C.  LIFECYCLE 

The  lifecycle  of  a  CoP  follows  natural  self-governed  iterations  controlled  by  the 
community.  CoPs  have  lifespans — they  emerge,  grow,  and  either  end  or  sustain.  One  sign 
of  a  successful  CoP  lifespan  is  if  “over  time  the  energy,  commitment  and  visibility 
provided  by  its  members  grows  until  it  becomes  institutionalized  as  a  core  value-added 
capability  of  the  sponsoring  organization”  [13].  The  model  seen  in  Figure  1  depicts  a 
composite  of  several  organizations  and  outlines  the  lifecycle  phases  of  communities. 


Figure  1.  Community  of  Practice  Fifecycle  Phases.  Adapted  from  [13]. 
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We  can  see  in  Figure  1  that  there  are  six  phases  within  the  lifecycle  of  a  CoP. 
These  phases  include:  “Inspire,  Design,  Prototype,  Launch,  Grow,  and  Sustain”  [13].  The 
first  phase,  inspire,  focuses  on  the  “process  of  exploration  and  inquiry,  identifies  the 
audience,  purpose,  goals,  and  vision  for  the  CoP”  [13].  The  second  phase,  design,  is 
where  we  “define  the  activities,  technologies,  group  processes,  and  roles  that  will  support 
the  CoP’s  goals”  [13].  The  third  phase,  prototype,  is  where  we  “pilot  the  community  with 
a  select  group  of  key  stakeholders  to  gain  commitment,  test  assumptions,  refine  the 
strategy,  and  establish  a  success  story”  [13].  The  fourth  step,  launch,  is  where  the  CoP  is 
introduced  to  a  “broader  audience  over  a  period  of  time  in  ways  that  engage  newcomers 
and  deliver  immediate  benefits”  [13].  The  fifth  step,  grow,  is  where  we  “engage  members 
in  collaborative  learning  and  knowledge  sharing  activities,  group  projects,  and 
networking  events  that  meet  individual,  group,  and  organizational  goals  while  creating  an 
increasing  cycle  of  participation  and  contribution”  [13].  Last,  the  sixth  step,  sustain, 
focuses  on  how  to  “cultivate  and  assess  the  knowledge  and  products  created  by  the  CoP 
in  order  to  inform  new  strategies,  goals,  activities,  roles,  and  technologies  for  the 
future”  [13].  Additionally,  the  blue  line  represents  the  positive  results  of  a  CoP  over  time 
if  the  right  amount  of  energy,  commitment,  and  visibility  is  given  to  all  phases,  whereas, 
the  red  line  shows  the  gradual  dissolution  of  the  CoP  as  a  result  of  neglect.  The  lifecycle 
phases  of  a  CoP  are  cyclic  and  may  change  depending  on  learning  and  reflection.  While 
not  displayed  in  Figure  1,  the  lifecycle  phases  of  communities  are  a  continuous  repeating 
cycle  of  trials  and  approaches,  and  each  step  may  be  revisited  at  any  time  to  implement 
and  try  different  methods. 

The  iterative  nature  of  CoP  development  suggests  that  it  takes  time  for  CoPs  “to 
emerge,  to  flourish,  and  to  become  productive”  [12].  Leadership  or  sponsors  cannot 
mandate  the  lifecycle  of  a  CoP,  nor  can  a  CoP  be  “managed  in  a  heavy-handed  way” 
[12].  If  the  CoP  is  managed  incorrectly,  it  will  more  than  likely  discourage  active 
participation  and  shorten  the  CoP’s  lifecycle.  CoPs,  as  previously  discussed  in  reference 
to  the  Chiefs  Mess,  are  “an  investment  in  the  organization’s  future,  not  a  quick  fix  to  be 
applied  for  the  sake  of  short-term  gain”  [12].  Thus,  the  mission,  vision,  and  purpose  are 
commensurable  with  the  lifecycle  of  a  CoP  and  will  evolve  with  the  community. 
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However,  opportunities  exist  for  CoP  sponsors  to  use  knowledge  of  a  CoP’s  lifecycle  to 
guide  the  positive  development  of  an  active  community. 

One  specific  way  sponsors,  champions,  and  stakeholders  may  move  a  CoP  toward 
growth  and  sustainment  is  by  designing  and  clearly  communicating  member  benefits. 
CoPs  often  grow  “based  on  the  increasing  benefits  individuals  or  organizations  accrue 
from  participating  in  the  activity”  [28].  Benefits  therefore  may  be  directly  linked  to  the 
lifecycle  of  a  CoP.  Defined  benefits  provide  potential  participants  of  a  CoP  with  reasons 
to  become  an  active  participating  CoP  member.  Benefits  accrue  from  participating  in  an 
activity,  and  may  include  rewards  or  recognition,  which  can  contribute  to  personal 
satisfaction.  Benefits  gained  by  participants  can  be  evaluated  and  examined  through 
multiple  iterations  and  prototypes  of  CoP  sponsored  events.  Furthermore,  successful 
CoPs  solve  important  problems  that  may  have  a  positive  impact  on  the  CoP  and  “retain  a 
substantial  percentage  of  its  members,  but  also  attract  new  ones”  [28].  Therefore, 
investment  in  defining  the  benefits  gained  by  sponsors,  champions,  stakeholders,  and 
members  of  a  CoP  may  positively  influence  the  lifecycle  of  a  sponsored  CoP. 

In  summary,  it  is  helpful  to  understand  a  CoP’s  lifecycle  phases  in  order  to 
establish  and  maintain  a  successful  community.  Members  of  the  community  have  the 
largest  role  in  determining  the  ultimate  success  or  failure  of  their  community,  but 
sponsors  can  help  define  membership  benefits,  which  directly  affect  CoP  longevity. 
Regular  assessments  should  be  made  of  the  CoP  in  order  to  determine  the  current 
lifecycle  phase  of  the  CoP  and  if  necessary  make  changes  to  the  CoP  structure  that 
influence  it  along  the  path  of  growth  and  sustainment. 

D.  CONCLUSION 

Successful  CoPs  are  defined  by  common  characteristics  to  include:  social 
interactions,  structure,  and  lifecycle.  Members  of  successful  CoPs  often  interact  in 
regular  and  frequent  social  interactions  either  face-to-face  or  virtually  as  a  means  to  share 
knowledge  and  collaborate.  The  structure  of  a  successful  CoP  consists  of  sponsorship 
support,  clear  and  visible  statements  that  define  the  CoP’s  goals  and  objectives,  inclusive 
membership  and  shared  identities  in  a  members’  practice  that  may  increase  long-term 
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CoP  commitment,  and  metrics  to  evaluate  and  measure  success.  When  creating  a  CoP  for 
Navy  programmers  and  developers,  momentum  and  vision  might  be  lost  at  any  particular 
phase,  which  could  result  in  the  designed  “CoP  not  achieving  the  overall  support  needed 
to  evolve  into  a  sustainable  entity”  [13].  Therefore,  the  remainder  of  this  thesis  examines 
techniques  to  design  a  community  with  these  key  characteristics  in  order  to  promote  its 
successful  establishment. 
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IV.  DESIGNING  A  COMMUNITY  OF  PRACTICE:  APPROACHES 

AND  TECHNIQUES 

Chapter  II  made  a  case  for  why  the  Navy  needs  a  CoP  for  programmers  and 
developers.  Chapter  III  highlighted  the  key  characteristics  of  successful  CoPs.  Chapter  IV 
focuses  on  how  to  create  and  design  a  CoP  and  explores  the  relationship  between  design, 
Design  Thinking,  and  human-centered  design.  Finally,  Chapter  IV  selects  a  design 
approach  that  will  be  used  for  making  recommendations  on  how  to  design  the  proposed 
Navy  community  of  practice. 

A.  WHAT  IS  DESIGN? 

Design  approaches  focus  on  end-results  from  the  viewpoint  of  users  and  it  can  be 
quite  effective  in  the  developmental  stages  of  a  CoP.  Design  is  a  process  that  has 
“evolved  rapidly  since  World  War  II”  [51].  The  word  design  takes  on  “a  variety  of  noun 
and  verb  meanings”  in  the  English  language,  and  “has  meanings  ranging  from  the 
abstract  conception  of  something  to  the  actual  plans  and  processes  required  to  achieve  it” 
[52].  According  to  Nigel  Cross,  British  academic,  design  researcher,  and  educator, 
“Everyone  can — and  does — design.  We  all  design  when  we  plan  for  something  new  to 
happen.  To  design  things  is  normal  for  human  beings,  and  ‘design’  has  not  always  been 
regarded  as  something  needing  special  abilities”  [53].  As  defined  by  Richard  Buchanan, 
world  leading  design  theorist,  design  is  “the  effort  of  people  to  make  the  products  that 
serve  us  in  our  everyday  lives”  [54].  Though  the  term  design  is  not  a  new  concept  or 
method,  it  has  evolved  over  the  centuries  in  its  use  and  definition. 

Historically  great  works  were  designed  by  single  minds  such  as  Homer, 
Shakespeare,  and  Leonardo  da  Vinci,  but  design  processes  increasingly  focus  on  the 
viewpoint  of  users  through  the  collaboration  of  many  minds.  According  to  Frederick 
Brooks,  a  designer  for  the  past  six  decades,  the  two  biggest  changes  in  design  since  the 
1990s  are  that  it  is  accomplished  “mostly  by  teams,  rather  than  individuals,  and  design 
teams  now  often  collaborate  by  using  telecommunications,  rather  than  by  being 
collocated”  [51].  Design  can  refer  to  the  shared  dialogue  between  individuals,  resulting  in 
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the  basis  for  thought  processes  and  collaborative  efforts.  It  can  be  viewed  as  a  “way  of 
knowing  through  thinking  and  doing,  as  well  as  from  giving  form  to  ideas  to  a  way  of 
doing  things”  [55].  Design  is  focused  on  the  collaboration  of  people  working  together  in 
efforts  to  create  tangible  products.  One  potential  barrier  of  incorporating  the  concept  of 
design  within  an  organization  is  the  misunderstandings  associated  with  what  design  is 
and  is  not. 

While  the  value  of  design  approaches  is  well  documented  in  Robert  Curedale’s 
book,  Design  Thinking  Processes  &  Methods  Guide,  organizations  often  dismiss  the  use 
of  design  due  to  a  lack  of  knowledge  about  “what  design  is  or  why  it  matters”  [49].  John 
Heskett,  design  scholar,  attempts  to  diffuse  this  misunderstanding  in  his  definition  of 
design,  “Design,  stripped  to  its  essence,  can  be  defined  as  the  human  nature  to  shape  and 
make  our  environment  in  ways  without  precedent  in  nature,  to  serve  our  needs,  and  give 
meaning  to  our  lives”  [56].  Organizations  can  benefit  from  the  practice  of  design  and  its 
ability  to  “tap  into  group  intelligence,  creativity,  and  ambition  to  make  a  meaningful 
impact  in  the  user’s  life,  both  functionally  and  emotionally”  [55].  Dispelling  misbeliefs 
about  design  is  imperative  to  the  adoption  of  design  approaches.  One  approach  to  better 
educate  organizations  on  the  purpose  and  benefits  of  design  approaches  is  through 
examination  of  successful  organizations  that  have  adopted  design  approaches.  Applying  a 
design  approach  will  add  value  to  launching  a  CoP  for  programmers  and  developers  with 
purpose. 

B.  WHAT  IS  DESIGN  THINKING? 

Design  Thinking  focuses  on  individuals’  needs  and  the  methods  required  to  create 
something  tangible  and  of  value  to  an  individual  or  an  organization.  Design  Thinking 
builds  on  “creativity  methods  of  the  1950s,  and  data  science  and  design  method 
movements  of  the  1960s”  [57].  Author  Herbert  Simon  explored  “the  notion  of  design  as  a 
way  of  thinking”  in  his  1969  book  The  Sciences  of  the  Artificial ,  but  the  term  Design 
Thinking  did  not  emerge  until  “the  1980s  with  the  rise  of  human-centered  design”  [57]. 
According  to  Robert  Curedale  [57],  author  of  Design  Thinking  Process  and  Methods ,  the 
term  Design  Thinking  was  not  used  in  design  literature  until  1987,  when  Peter  Rowe, 
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author  and  contributor  to  the  design  field,  released  his  book  Design  Thinking  [57].  Over 
the  past  thirty  years  the  term  has  evolved  to  incorporate  ideas  from  numerous  design 
methodologies  and  movements  [57]. 

In  the  1980s,  revealed  design  began  to  focus  increasingly  on  understanding  the 
needs  of  users  and  organizations.  Curedale  documents  how  practitioners  of  Design 
Thinking  began  incorporating  ideas  from  the  user-centered  design  movement.  In  the 
1990s,  IDEO,  an  “award-winning  global  design  firm,”  was  among  the  founders  of  the 
design  thinking  movement  [58].  Shortly  thereafter  in  the  early  2000s,  a  large  personal 
donation  was  made  to  the  Stanford  d. school,  which  “pioneered  the  teaching  of  Design 
Thinking”  [57], 

Design  Thinking  is  taught  and  used  nationwide  in  both  schools  and  communities. 
It  often  consists  of  three  key  tenets:  empathy,  collaboration,  and  accelerated 
learning  [55].  Additionally,  Design  Thinking  can  be  a  valuable  part  of  an  innovation 
process  [55].  At  the  Stanford  d.school,  the  Design  Thinking  process  “focuses  on  needs 
through  finding,  understanding,  creating,  thinking,  and  doing.  At  the  core  of  this  process 
is  a  bias  towards  action  and  creation:  by  creating  and  testing  something,  you  can  continue 
to  learn  and  improve  upon  your  initial  ideas”  [59].  IDEO  uses  Design  Thinking  to  “bring 
together  what  is  desirable  from  a  human  point  of  view  with  what  is  technologically 
feasible  and  economically  viable”  [58].  The  following  sections  explore  the  three 
identified  tenets  of  Design  Thinking  and  their  applicability  to  innovation. 

1.  Empathy 

Design  Thinking  focuses  on  understanding  the  needs  of  users  accomplished 
through  research  that  uses  empathetic  approaches.  In  design,  empathy  may  be  defined  as 
“identifying  with  others  and  adopting  their  perspective”  [57].  Empathy  is  important 
because  of  the  respect  gained  through  observation  and  understanding  of  other’s  points  of 
view  and  ideas.  Empathetic  approaches  serve  as  both  a  “source  of  inspiration  to  the 
designer  and  aid  in  discovering  user  insights  and  unarticulated  needs”  [55].  Empathetic 
discovery  may  be  accomplished  through  interviews,  surveys,  observations,  and 
ethnographic  methods  to  include:  “watching,  listening,  discussing,  and  seeing  in  order  to 
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better  understand  user  needs”  [55].  The  purpose  of  empathetic  approaches  is  not  to 
persuade  a  group  of  individuals,  but  rather  to  seek  and  understand  their  point  of  view,  and 
build  trust.  Trust  built  among  users  further  encourages  collaboration  in  designing  a 
product  suitable  to  the  needs  of  all  users. 

2.  Collaboration 

The  second  tenet  of  Design  Thinking,  collaboration,  helps  organizations  move 
toward  “radical  innovation,  rather  than  incremental  improvement,  and  seeks  added  value” 
in  the  design  of  a  product  [55].  Collaboration  occurs  between  users  and  through  the 
formation  of  teams  or  groups.  Design  Thinking  combined  with  the  added  benefits  of 
collaboration  plays  a  valuable  role  in  “unlocking  new  opportunities”  [55].  Often  “the  best 
way  to  get  a  good  idea  is  to  get  a  lot  of  ideas,”  which  often  happens  through  active 
collaboration  [60].  As  highlighted  in  Chapters  II  and  III,  collaboration  helps  organize 
individuals  to  generate  new  ideas  and  share  information  to  produces  tangible  results. 
Collaboration  using  a  Design  Thinking  approach  can  help  individuals  unlock  creativity, 
produce  new  ideas,  and  generate  solutions  to  problems.  Collaboration  is  an  important  part 
of  creating  an  innovation,  defined  as  “an  adoption  of  new  practice  in  a  community”  [61]. 
Collaboration  is  inherent  throughout  the  innovation  process,  including  sensing  a 
community’s  concerns,  envisioning  what  is  needed  to  address  those  concerns,  moving  to 
adoption,  and  sustainment  of  a  change. 

3.  Accelerated  Learning 

Collaboration  may  lead  to  accelerated  learning  through  iterations  of  innovative 
visualization,  experiments,  and  prototypes  [55].  Accelerated  learning  is  useful  for 
proving  useable  feedback  in  order  to  assess  the  success  of  a  design.  In  order  to  attain 
radical  innovation  through  accelerated  learning  “the  more  experimentation  the 
better”  [55].  Additionally,  quick  and  simple  prototypes  are  ideal,  because  they  provide 
rapid  feedback  in  regards  to  implementation,  resources  need,  funds,  and  development. 
One  goal  of  accelerated  learning  through  prototypes  is  to  “fail  quickly  and  frequently  so 
that  learning  can  occur”  [55].  It  is  often  recommended  to  “make  simple  prototypes  and 
constantly  test  ideas  as  early  as  possible”  throughout  the  design  of  something  [55]. 
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Prototypes  should  be  used  to  guide  and  improve  the  design.  Expectations  should  be  made 
that  change  will  need  to  occur  throughout  the  phases  of  design.  The  key  to  accelerated 
learning  is  the  Design  Thinking  framework  that  further  supports  the  iterative  cycle  of 
prototypes,  the  innovation  process,  and  the  tools  needed  to  achieve  design  of  a  product 

The  Design  Thinking  approach  is  iterative,  not  linear.  Iterative  processes  are  key 
to  reducing  risk  of  failure  in  addition  to  accelerating  organizational  learning  [55].  As  seen 
in  Figure  2,  this  Design  Thinking  framework  combines  “classic  creative  problem  solving 
with  design  methodologies”  [59]. 


1.  Discovery 


Choose  an  affirmative,  strategic  topic. 
Gather  data.  Understand  &  empathize 
with  unmet  needs. 


2.  (Re)Frame  opportunity 

Look  for  patterns  &  insights. 
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5.  Evaluate/Refine  ideas 

What  is  desirable,  feasible,  viable 

about  your  ideas?  What  are  the  constraints? 
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Figure  2.  A  Framework  for  Design  Thinking.  Source:  [59]. 

This  framework  highlights  the  continuous  nature  of  Design  Thinking.  Instead  of  a 
linear  representation,  it  shows  a  repeating  cycle  of  trials  and  approaches,  to  include 
prototypes  and  tests.  Successful  design  will  undergo  multiple  iterations  and  may  revisit 
any  phase  of  iteration  as  needed.  The  objective  of  this  Design  Thinking  framework  is  “to 
make  the  intangible  become  tangible”  [55]. 
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Design  Thinking  presents  a  “very  clear  shift  towards  a  more  creative  and  more 
collaborative  way  of  working — one  in  which  intuition  counts  heavily,  experimentation 
happens  fast,  failures  along  the  way  are  embraced  as  learning,  business  strategy  is 
integrated,  and  more  relevant  solutions  are  produced”  [55].  Additionally,  “Design 
Thinking  is  driven  by  intelligence  that  embraces  innovation  and  gives  an  organization  the 
freedom  to  explore  multiple  ways  to  solve  problems  -  and  discover  the  option  that  best 
delivers  competitive  advantage”  [55].  Design  Thinking  creates  and  considers  multiple 
options,  refines  selected  courses  of  actions,  and  plans  for  their  execution,  while  efforts 
remain  focused  on  resolving  problems  through  the  use  of  design  and  test  prototypes. 
Design  Thinking  is  an  approach  that  involves  a  “toolkit  of  methods  that  can  be  applied  to 
different  problems  by  cross  disciplinary  groups  or  by  individuals”  [57]. 

To  help  designers  accelerate  learning,  Design  Thinking  utilizes  a  toolkit  of 
methods  to  “define  the  problem  and  implement  solutions,  always  with  the  needs  of  the 
user  demographic  at  the  core  of  concept  development”  [59].  Design  Thinking  is  an 
“approach  that  supports  innovation”  [61].  It  emphasizes  “observation,  collaboration,  fast 
learning,  visualization  of  ideas,  rapid  concept  prototyping,  and  concurrent  analysis,  which 
ultimately  influences  innovation  and  strategy”  [55].  The  tenets  of  Design  Thinking, 
empathy,  collaboration,  and  accelerated  learning,  align  with  the  key  characteristics  of 
successful  CoPs,  identified  in  Chapter  III. 

C.  HUMAN-CENTERED  DESIGN 

Human-centered  design  is  a  “Design  Thinking  approach  that  is  people-oriented” 
[62].  Human-centered  design  “started  to  evolve  around  the  late  1990s”  [62].  Prior  to  this, 
“human-centered  design  and  user-centered  design  were  often  interchangeable  terms 
regarding  the  integration  of  end  users  within  a  design  process”  [62].  Human-centered 
design  author,  William  Rouse,  describes  human-centered  design  as  philosophical:  “Roles 
of  humans  in  complex  systems,  enhancing  human  abilities,  aid  to  overcome  human 
limitations,  and  foster  user  acceptance”  [63].  Over  the  past  two  decades  human-centered 
design  has  “transformed  from  a  method  to  a  mindset,  aimed  to  humanize  the  design 
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process  and  empathize  with  stakeholders,  and  further  re-introduced  Design  Thinking  as  a 
mindset  used  for  interpreting  problems”  [63]. 

The  mindset  of  human-centered  design  is  associated  with  “a  deep  understanding 
of  the  people  for  whom  the  product  is  intended”  through  means  of  observation  [64].  As 
defined  by  Grant  Young,  international  designer,  human-centered  design  takes  the 
“perspective  of  the  end-users”  and  places  their  needs  at  the  “center  of  the  design  process” 
[65].  Furthermore,  a  “human-centered  approach  fuels  the  creation  of  products  that 
resonate  more  deeply  with  an  audience — ultimately  driving  engagement  and 
growth”  [66].  In  2007,  a  MIT  Business  School  study  noted,  “70%  to  80%  of  new  product 
development  that  fails  does  so  not  for  lack  of  advanced  technology,  but  because  of  a 
failure  to  understand  users’  needs”  [67].  Therefore  it  is  imperative  to  observe  and 
understand  the  needs  of  users  when  designing  something. 

AirBnB  and  Pinterest  are  examples  of  successful  companies  that  used  design- 
driven  approaches  to  promote  and  flourish  in  the  business  sector  [66].  Their  successes 
may  be  attributed  to  their  focus  on  user  needs,  tailored  website  design  to  address  these 
needs,  and  advertisement  that  targeted  the  intended  audience  in  order  to  encourage 
participation,  growth,  and  financial  gains  to  further  company  success.  In  order  to 
encourage  customer  feedback  and  support,  the  design  approach  must  “cultivate  a  shared 
ownership  of  ideas,  as  new  ideas  are  the  team’s  idea,  not  an  individual’s”  [66].  These 
approaches  of  shared  ideas  proved  successful  to  AirBnB  and  Pinterest,  which  have  used 
human-centered  design  approaches  to  further  grow  and  sustain  their  businesses  [66]. 

The  mindset  of  human-centered  design  correlates  to  innovation,  because  of  the 
shift  in  thinking  from  “me”  to  “we”  when  establishing  the  best  possible  solutions  for 
solving  a  problem.  Vijay  Kumar,  distinguished  college  professor,  correlates  “human- 
centered  design  to  innovation,  noting  that  in  order  to  create  innovations  that  have  a  good 
fit  with  users,  the  designer’s  focus  needs  to  shift  from  products  that  people  use,  to  what 
those  people  do — their  behaviors,  activities,  needs,  and  motivations”  [68].  The  designers 
of  IDEO  agree,  “Design  Thinking  is  a  human-centered  approach  to  innovation  that  draws 
from  the  designer’s  toolkit  to  integrate  the  needs  of  people,  the  possibilities  of 

technology,  and  the  requirements  for  business  success”  [58]. 
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In  summary,  human-centered  design  is  “not  about  claiming  credit,  as  no  good  idea 
comes  from  just  one  person,  it  is  about  the  quality  of  the  idea  and  success  of  the  team” 
[66].  A  human-centered  design  approach  focuses  on  the  needs  of  the  intended  audience, 
made  through  observations  and  inquiries  [58].  In  order  to  promote  and  encourage  creative 
innovation  among  consumers,  consumers  must  understand  that  their  ideas  and  opinions 
are  valued.  Team  focused  approaches  often  encourage  the  free  flow  of  ideas.  A  human- 
centered  design  approach  is  ideal  for  designing  a  CoP  because  its  collaborative  team- 
focused  nature  encourages  innovative  thinking  by  all  of  those  interested  in  achieving  the 
desired  result. 

D.  A  DESIGN-THINKING  /  HUMAN- CENTERED  DESIGN  APPROACH 

Design  Thinking  coupled  with  human-centered  design  is  people-oriented  and 
focused  on  the  needs  of  consumers.  As  described  by  IDEO  CEO,  Tim  Brown,  “Design 
Thinking  provides  an  innovative  approach  that  is  human  centered  in  an  effort  to  solve 
problems  based  on  the  needs  of  the  people  at  the  center  of  the  design”  [58].  IDEO 
remains  one  of  the  leaders  in  human-centered  design  approaches,  but  other  design 
consultancies  occupy  the  same  philosophical  space. 

The  LUMA  Institute,  a  “spin-off  of  the  successful  Pittsburgh-based  design 
consultancy  MAYA  Design,  has  fully  embraced  the  notion  of  design,  design  thinking, 
and  human-centered  design”  [69].  The  Institute  “equips  individuals,  teams,  and 
organizations  to  accelerate  innovation”  [70].  LUMA’s  approach  “is  based  on  the  use  of 
techniques  which  communicate,  interact,  empathize,  and  stimulate  the  people  involved, 
obtaining  an  understanding  of  their  needs,  desires,  and  experiences”  [71].  The  LUMA 
Institute  promotes  human-centered  design  training  through  workshops  and  training 
offered  worldwide.  Their  handbook  provides  recommended  tools  and  techniques  that 
may  be  used  for  human-centered  design. 

The  LUMA  Institute  [72]  is  an  educational  organization  that  provides  designers 
with  an  assortment  of  tools  and  techniques  so  they  may  create  purposeful  and  valuable 
products  centered  on  the  needs  of  users.  The  tools  and  techniques  provided  in  the  LUMA 
Institute  handbook  focus  on  methods  associated  with  the  key  tenets  of  Design  Thinking 
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to  include:  empathy,  collaboration,  and  accelerated  learning.  As  seen  in  Figure  3,  their 
approach  is  composed  of  three  phases:  looking,  understanding  and  making  [72]. 
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Figure  3.  The  LUMA  Fluman-centered  Design  Approach.  Source:  [72]. 


The  LUMA  Institute  human-centered  design  approach  aligns  with  and  simplifies 
standard  Design  Thinking  processes  by  organizing  into  three  phases  and  providing 
techniques  to  help  designers  understand  how  to  progress. 

1.  Phase  I — Looking 

The  purpose  of  the  looking  phase  is  “inquiry,  analysis,  and  design  through 
ethnographic,  participatory,  and  evaluative  research  in  order  to  identify  the  audience, 
purpose,  goals,  and  vision  for  the  community”  [72].  These  research  methods  may  be 
conducted  through  interviews,  observations,  surveys,  and  activities.  Traditional 
ethnographic  research  includes  “participant  and  nonparticipant  observation,  focus  on 
natural  settings,  use  of  participant  constructs  to  structure  the  research,  and  investigator 
avoidance  of  purposive  manipulation  of  study  variables”  [73].  Ethnographic  approaches 
are  “common  in  sociology  and  anthropology,  but  are  used  to  some  extent  by  all  social 
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science  disciplines”  [73].  Ethnographic  techniques  are  used  to  gather  information  through 
direct  or  indirect  dialogue  in  order  to  obtain  a  holistic  understanding  of  consumer  needs. 
Ethnographic  research  further  “develops  empathy  for  the  point  of  view  of  the  user”  [57]. 
One  disadvantage  of  ethnographic  research  is  that  results  are  often  “regarded  as 
unreliable  and  lacking  in  validity  and  generalizability”  [73].  Regardless,  empathy  is 
necessary  because  “in  order  to  get  to  new  solutions,  you  have  to  get  to  know  different 
people,  different  scenarios,  and  different  places”  [74]. 

Designers  use  specific  techniques  in  the  looking  phase  to  gather  valuable 
information  that  helps  them  understand  users’  in  their  natural  working  environments. 
Recommended  techniques  encourage  the  designer  to  immerse  themselves  in  the  users’ 
lives  and  practice,  because  of  the  insight  gained  from  this  in-context  interaction  [74]. 
Participatory  techniques  may  involve  potential  community  stakeholders  in  order  to  create 
and  host  activities  that  seek  to  “reveal  individualistic  thoughts  and  ideas,  level  of 
prioritization,  key  challenges,  values,  requirements,  and  opportunities  for  improvement” 
[72].  Participatory  research  is  critical  to  maintaining  creativity  and  energy  among 
collaborating  teams  in  order  to  refine  the  design  process  based  on  feedback  [74]. 
Evaluative  techniques  and  activities  encourage  non-attribution  of  inputs  and  ideas,  while 
providing  a  forum  for  individuals  to  provide  constructive  feedback.  One  goal  of  the 
looking  phase  is  to  gather  both  subjective  and  objective  feedback  from  consumers  in 
order  to  better  understand  their  needs,  and  incorporate  this  feedback  into  the  overall 
design. 


2.  Phase  II — Understanding 

The  understanding  phase  focuses  on  prototyping,  which  takes  into  consideration 
people,  systems,  patterns,  priorities,  and  problem  framing  [72].  The  tools  and  techniques 
provided  in  this  phase  aid  in  determining  the  key  characteristics  of  successful 
communities  as  highlighted  in  Chapter  III,  to  include  identifying  sponsors,  structural 
designs,  and  strategies  that  aim  for  a  sustainable  lifecycle.  Furthermore  the  understanding 
phase  assists  the  designer  in  determining  “the  needs  and  goals  of  people,  and  methods  to 
help  identify  benefits  gained  from  this  phase”  [72].  It  also  provides  the  designer  the 
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opportunity  to  “make  sense  of  a  vast  amount  of  data,  and  identify  opportunities  for 
design”  [74].  Furthermore,  analysis  of  results  from  the  looking  phase  may  identify 
patterns  and  themes  that  can  be  used  to  design  potential  prototypes.  Upon  successive 
iterations  of  prototypes,  a  product  may  then  be  created. 

Iterative  approaches  are  key  to  solving  problems  because  “it  makes  feedback  from 
the  people  we’re  designing  for  a  critical  part  of  how  a  solution  evolves”  [74].  Continuous 
iterations  of  prototypes  allows  for  improvement  and  refinement  of  the  work,  and  further 
provides  opportunities  to  unlock  creativity  and  arrive  more  efficiently  to  successful 
solutions  [74].  Rarely  is  something  designed  correctly  the  first  time.  The  understanding 
phase  provides  appropriate  techniques  to  be  explored  in  order  to  arrive  at  a  solution  for 
future  adoption.  The  end  goal  of  the  understanding  phase  is  to  combine  the  collaborative 
efforts  and  innovative  ideas,  conduct  successive  prototypes,  and  create  robust  solutions 
for  implementation  into  future  designs. 

3.  Phase  III — Making 

The  making  phase  focuses  on  the  methods  for  envisioning  future  possibilities, 
including  how  to  launch,  grow,  and  sustain  a  community  [72].  While  the  first  two  phases 
provide  techniques  and  tools  to  better  understand  why  something  is  being  designed,  the 
third  phase  is  subjective  and  exploratory  in  the  actual  design.  One  way  to  design 
something  is  by  examining  previous  successful  examples  from  the  same  domain  as  the 
one  currently  being  created.  Another  common  approach  to  design  is  the  use  of  checklists. 
This  phase  assists  the  designer  in  concept  ideation,  modeling  and  prototyping,  and  design 
rationale,  but  ultimately  it  is  up  to  the  designer  as  to  how  they  make  something  [72]. 

The  goal  of  the  making  phase  is  actual  implementation  of  solutions.  This  is  the 
part  of  the  process  where  ideas  get  piloted.  If  throughout  the  design  process,  “the  very 
people  you  were  looking  to  serve  were  kept  at  the  heart  of  the  process”  then  in  theory  the 
solution  will  be  a  success  [74].  As  solutions  are  implemented,  it  is  also  important  to 
consider  designing  a  timeline  or  plan  of  action  for  sustainment  of  the  design  project. 
Successive  analysis  of  solutions  will  need  to  be  executed  to  evaluate  if  the  solution  is 
working  or  providing  the  results  expected.  This  phase  is  when  a  designer  further  assesses 
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time,  manpower,  and  funds  needed  to  sustain  the  designed  product.  Finally,  during  the 
making  phase,  collaboration  among  designers,  stakeholders,  and  members  should  occur 
in  order  to  determine  the  desired  end-state.  These  milestones  will  help  “keep  the  design 
on  course  and  give  the  team  something  to  work  toward”  [74]. 

E.  CONCLUSION 

Design,  Design  Thinking,  and  human-centered  design  approaches  inspire 
creativity  and  innovation,  while  providing  designers  with  the  tools  and  techniques  to 
effectively  design  a  sustainable  and  valuable  product.  These  approaches  focus  on  the 
deepened  understanding  of  consumer  needs  and  actions  taken  to  create  something  that 
reflects  this  discovery.  The  process  to  obtain  this  desired  end  state  is  a  developed 
empathy  for  users’  needs,  collaboration  that  inspires  innovation,  and  prototyping  through 
successive  iterations  and  analysis.  This  thesis  asserts  that  a  design  approach  that  utilizes 
Design  Thinking  and  human-centered  design  is  most  appropriate  for  the  creation  of  a 
Navy  CoP  for  programmers  and  developers  because  of  its  focus  on  users’  needs.  Chapter 
V  will  elaborate  on  how  the  LUMA  Institute  human-centered  design  approach  was 
implemented  to  make  recommendations  for  launching  and  prototyping  a  Navy  CoP. 
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V.  DESIGNING  A  COMMUNITY  OF  PRACTICE  FOR  NAVY 
PROGRAMMERS  AND  DEVELOPERS 


Design  approaches  can  be  used  to  create  something  with  purpose  and  intent  in 
order  to  solve  a  problem.  Furthermore,  Design  Thinking  and  human-centered  design 
approaches  benefit  from  deepened  empathy  for  users’  needs  in  order  to  design  something 
both  tangible  and  of  value  to  the  users.  Chapter  V  examines  how  the  three  phases  of  the 
LUMA  Institute  human-centered  design  approach  ( looking ,  understanding,  and  making ) 
were  implemented  in  order  to  make  design  recommendations  to  launch  a  Navy  CoP  for 
programmers  and  developers. 

A.  PHASE  I— LOOKING 

Following  the  LUMA’s  recommended  techniques  for  the  looking  phase, 
ethnographic  and  evaluative  research  was  employed  to  observe  and  discover  users’  needs. 
The  author  explored  the  techniques  by  taking  advantage  of  the  proximity  and  audience  of 
HACKtheSKY,  the  U.S  Navy’s  first  hackathon  focused  on  the  practices  of  programmers 
and  developers.  A  hackathon  is  an  event  where  large  groups  of  individuals  meet  in  order 
to  engage  in  collaborative  efforts  to  solve  problems  [75].  During  this  three-day 
cybersecurity  event  interviews,  observations,  and  surveys  were  conducted  among 
consenting  participants.  Data  obtained  from  this  research  were  subsequently  analyzed  and 
in  the  understanding  phase  and  recommendations  for  the  design  of  a  Navy  CoP  for 
programmers  and  developers  are  proffered  in  the  making  phase. 

1.  HACKtheSKY 

The  HACKtheSKY  Navy  hackathon  was  hosted  by  the  Department  of  the  Navy 
Office  of  Strategy  and  Innovation  and  the  Naval  Postgraduate  School  on  June  24-26, 
2016,  in  San  Francisco,  California  at  Galvanize,  a  startup  incubator.  The  event  theme  was 
based  on  the  world  record  set  at  NPS  in  2015  for  flying  the  largest  swarm  of  drones  ever 
controlled  by  a  common  code  base  [76].  The  event  included  three  different  sub-events: 
Hackathon,  Future  of  Autonomy  Workshop,  and  Topcoder  Rapid  User  Experience 
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(RUX)  challenge.  The  hackathon  was  oriented  toward  programmers  and  developers,  as 
well  as  user-experience  designers,  unmanned  aerial  vehicle  hobbyists,  and  data  scientists. 
The  future  of  autonomy  workshop  was  oriented  toward  technical  and  non-technical 
participants  looking  to  help  develop  the  autonomy  roadmap  for  the  Navy.  The  Topcoder 
RUX  challenge  mimicked  the  hackathon  focus,  but  took  advantage  of  a  global  audience 
in  order  to  produce  a  larger  pool  of  tangible  solutions. 

HACKtheSKY  hosted  a  total  of  220  hackers,  coders,  designers,  data  scientists, 
and  developers  from  around  the  world  over  the  course  of  three  days  [76].  The  hackathon 
was  open  to  all  military  (active  duty  and  reserve),  DOD,  and  industry  civilians  of  all  skill 
levels  and  interests.  There  were  17  organizations  represented  from  the  Navy,  six  from 
DOD,  and  15  organizations  and  start-ups  represented  from  the  technology  industry  and 
academia  [76].  Of  the  151  on-site  participants,  there  were  36  military  personnel,  113 
industry  civilians,  and  two  unaffiliated  with  any  organization.  Additionally,  of  the  151 
on-site  participants  133  were  U.S.  citizens,  16  non-U. S.  citizens,  and  two  of  origins 
unknown  [77].  HACKtheSKY  provided  a  group  of  possible  participants  to  observe  while 
trying  out  techniques  from  the  looking  phase.  Additionally,  HACKtheSKY  provided  an 
opportunity  to  observe  and  interview  programmers,  developers,  and  Design  Thinkers  in 
their  near-natural  environments. 

2.  Ethnographic  Research 

One-on-one  interviews  and  passive  “fly-on-the-wall”  observations  were  used  to 
obtain  a  holistic  understanding  of  participants’  needs.  Of  those  interviewed,  half  were 
active  duty  military  and  half  were  civilians  from  the  technology  industry.  Of  those 
participants  from  industry,  several  of  them  had  prior  military  experience.  Participants 
shared  their  experiences  and  thoughts  in  order  to  assist  in  better  formulate  ideas  and 
concepts  for  how  to  best  design  a  Navy  CoP  for  programmers  and  developers. 

a.  Interviews 

Interviews  provide  an  opportunity  to  gather  valuable  information  from  individuals 
in  order  to  better  understand  needs,  desires,  and  opinions.  Interviews  conducted  at 

HACKtheSKY  were  unscripted  and  often  on  the  fly,  with  no  pre-planned  questions. 
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Interviews  were  held  external  to  participant’s  day-to-day  operations  as  to  not  affect  work 
schedules.  They  were  conducted  one-on-one  or  as  a  group,  which  enabled  dialogue  and 
the  free  exchange  of  information  between  participants.  The  demographic  of  individuals 
interviewed  included  75%  Navy  programmers  and  developers  (officer,  enlisted,  civilian, 
and  contractor)  and  25%  civilians.  Of  the  active  duty  Navy  participants,  23%  of  them 
worked  as  programmers  or  developers  in  cyber  operations  related  billets,  and  the  other 
77%  engaged  in  the  practice  as  a  hobby.  Of  the  civilians  interviewed,  66%  of  them 
worked  in  programmer  and  developer  related  jobs,  and  the  other  34%  engaged  in  the 
practice  as  a  hobby.  Of  the  active  duty  Navy  and  civilian  participants,  29%  performed  in 
billets  within  the  formal  cyber  workforce,  12%  worked  within  industry,  59%  were 
hobbyists,  and  0%  of  participants  were  from  the  informal  cyber  workforce. 

Interviews  revealed  a  shared  sense  of  need  and  support  for  a  Navy  CoP  for 
programmers  and  developers.  Interviews  also  provided  the  following  ideas,  concerns,  and 
recommendations  for  the  creation  and  design  of  a  Navy  CoP  for  programmers  and 
developers: 

•  Participants  expressed  a  strong  need  and  desire  to  collaborate  and  bring 
together  like-minded  programmers  and  developers  from  the  military, 
technology  industry,  and  commercial  sector. 

•  Participants’  supported  the  idea  of  continued  Navy-sponsored 

cybersecurity  themed  hackathons.  Future  hackathons  should  be  challenge- 
driven  instead  of  driven  by  operational  needs,  in  order  to  encourage 
excitement  and  energy  among  participants. 

•  Face-to-face  interactions  help  to  build  trust,  share  knowledge,  and 
improve  skills. 

•  Hackathons  should  encourage  participation  from  individuals  who  are  not 
necessarily  programmers  and  developers,  which  would  encourage  the  free 
flow  of  ideas  and  sharing  of  knowledge  from  a  diverse  spectrum  of 
mindsets  in  order  to  solve  real-time  Navy  problems. 

•  Future  hackathons  should  be  hosted  in  technology  industry  popular  cities 
such  as  Seattle,  Boston,  or  Austin,  because  of  the  diverse  levels  of 
expertise  and  concentration  of  programmers  and  developers  in  these  areas 
who  could  help  expand  the  boundaries  of  Navy  cybersecurity  practices. 
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•  Military  personnel  should  attend  more  hackathons  offered  throughout  the 
technology  industry  in  order  to  help  individuals  expand  their  social 
network  and  further  develop  and  improve  programming  and  developing 
skills. 

•  Further  investigation  should  be  made  into  existing  programmer  and 
developer  communities  (e.g.,  New  Tech  Seattle  and  VRhackathon)  in 
order  to  understand  how  they  remain  effective  and  successful  within 
industry. 

In  conclusion,  the  benefits  of  conducting  interviews  was  the  information  gained 
directly  from  programmers  and  developers  who  wanted  to  share  information,  challenge 
preconceptions,  and  helped  to  deepen  empathy  for  the  challenges  programmers  and 
developers  face  in  their  work.  Through  one-on-one  and  group  interviews,  participants 
expressed  a  desire  to  expand  the  programmer  and  developer  communities,  including  the 
formal  and  informal  cyber  workforces  and  industry.  HACKtheSKY  proved  to  be  a 
success.  Participants  expressed  desire  for  NPS  to  host  future  hackathons  in  support  of 
collaboration  toward  creating  solutions  to  military  cybersecurity  issues. 

b.  Fly-on-the-Wall  Observations 

Fly-on-the-wall  observations,  a  LUMA-defined  term  and  method,  provide  an 
opportunity  to  observe  participants  in  an  unobtrusive  manner,  while  having  the  least 
amount  of  impact  on  the  flow  of  participants’  regular  activities.  This  method  allowed  the 
author  to  pay  careful  attention  to  individuals’  tasks  and  workflow,  communication  styles, 
and  collaboration  techniques.  The  author  assumed  the  role  of  an  objective  bystander, 
continuously  walking  and  roaming  through  both  the  future  of  autonomy  and  hackathon 
workspaces  over  the  course  of  the  weekend,  while  taking  notes  and  making  observations 
of  both  groups  of  participants.  Overall  observations  include: 

•  The  majority  of  participants  exerted  an  energetic  feeling  and  excitement 
expressed  in  their  collaboration  and  conversations,  which  carried  on 
throughout  the  weekend. 

•  Teams  engaged  in  open  collaborative  conversations  and  appeared  to 
encourage  the  free-flow  of  ideas.  Furthermore,  collaboration  seemed  to 
flow  seamlessly  within  most  teams. 
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•  Design  Thinkers  were  coached  on  how  to  present  their  ideas  in  a 
collaborative  way,  which  was  reflected  in  their  energetic  group 
participations.  The  hackers  who  were  not  coached  on  how  to  present,  and 
while  possibly  not  related,  overall  the  hacker’s  gave  less  energetic 
presentations  than  those  who  were  coached  on  providing  effective 
presentations. 

•  The  forum  provided  a  useful  venue  to  encourage  innovative  thinking  in 
order  to  formulate  more  creative  strategies  to  solve  Navy  problems.  One 
participant  stated  that  for  their  35  years  in  the  Navy,  they  were  always  told 
to  think  or  stay  in  the  box. 

•  Many  participants  were  critical  of  the  Navy’s  “archaic  technology,”  and 
commented  that  the  Navy  could  benefit  from  leveraging  the  latest 
technological  advances  of  industry,  and  through  collaboration  and  sharing 
knowledge  with  industry  may  improve  Navy  processes.  They  also  made 
comments  about  the  development  of  cybersecurity  tools  and  their 
functionality  in  that  it  is  often  lacking  within  the  DOD  compared  to  the 
innovative  developments  found  in  the  technology  industry. 

•  The  inclusion  of  students  as  young  as  mid-  to  late-teens  in  HACKtheSKY 
provided  value  to  the  community.  These  teenagers  learned  about  how  they 
could  or  might  contribute  to  their  nation  by  collaborating  to  solve  Navy 
problems.  Furthermore,  it  allowed  them  to  expand  their  social  network  at  a 
young  age,  which  may  attribute  to  continued  interest  and  improvement  in 
their  programming  and  developing  skills. 

•  It  was  valuable  to  expose  programmers  and  developers  to  Design  Thinking 
techniques  in  order  to  expand  their  knowledge  and  introduce  new 
innovative  ways  of  solving  problems. 

In  summary,  fly-on-the-wall  observations  provided  thought  provoking  ideas  to 
use  for  further  analysis  when  developing  recommendations  for  how  to  design  a  Navy 
CoP  for  programmers  and  developers.  Future  of  autonomy  and  hackathon  participants 
provided  different  mindsets  and  approaches  to  similar  problems.  This  looking  phase 
technique  provided  a  method  to  observe  participants  in  an  unobtrusive  manner,  but  still 
collect  value  insight  from  their  daily  interactions  and  collaborative  efforts. 

3.  Evaluative  Research 

Surveys  are  one  method  recommended  by  the  LUMA  Institute  handbook  for  how 
to  conduct  evaluative  research  in  order  to  collect  candid  feedback  from  participants.  Pre¬ 
drafted  surveys  were  created  and  approved  for  use  at  HACKtheSKY.  Pre-drafted  surveys 
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included  pre-hackathon  and  post-hackathon  questions,  as  well  research  interview 
questionnaires  focused  specifically  on  the  creation  of  a  CoP  for  Navy  programmers  and 
developers.  Furthermore,  research  interview  questionnaires  were  divided  into  two 
categories  (active  duty  military  and  non-active  duty  military),  and  distributed  to 
consenting  participants  based  on  their  identified  affiliation  as  a  programmer,  developer, 
or  Design  Thinker.  The  surveys  provided  an  additional  source  of  documentation  for  the 
author’s  recommendations  on  how  to  improve  processes  and  procedures. 

The  purpose  of  HACKtheSKY  surveys  was  to  solicit  and  crowdsource  opinions, 
thoughts,  suggestions,  and  critiques  regarding  a  Navy  CoP  for  programmers  and 
developers,  regardless  of  individual  skill  level.  Thirty-one  signed  consent  forms  were 
collected.  Of  those  thirty-one  signed  consent  forms,  sixteen  participants  completed 
requested  surveys.  Of  those  sixteen  participants  who  completed  surveys,  62%  were 
active-duty  military,  31%  worked  in  the  technology  industry,  19%  were  veterans,  and  6% 
were  Navy  reservists.  (Note:  there  were  no  overlaps  among  these  groups.)  The  following 
section  summarizes  survey  results  and  highlights  key  takeaways.  Appendixes  A-D 
provide  in-depth  analysis  of  all  four  surveys,  questions,  and  responses  received. 

a.  Pre-hackathon  Survey 

The  pre-hackathon  survey  consisted  of  fifteen  questions.  The  intent  of  this  survey 
was  to  collect  demographic  information  about  individuals  attending  HACKtheSKY,  gain 
a  better  understanding  of  their  self-assessed  individual  skill  level  as  a  programmer  or 
developer,  and  insight  as  to  what  their  expectations  of  the  event  were.  Below  are  the 
summarized  results  from  the  pre-hackathon  survey. 

The  vast  majority  of  pre-hackathon  survey  participants  consisted  of  active  duty 
military  officers,  but  also  included  several  enlisted  participants,  and  two  civilians,  each 
with  approximately  of  nine  years’  experience  as  enlisted  cryptologists.  Collectively 
participants  held  some  form  of  college  education,  but  lacked  formal  military,  civilian 
training,  or  education  related  to  software  and  firmware  programming  and  developing.  Of 
those  participants  who  completed  this  survey,  only  a  third  of  them  performed  in 
programmer  or  developer  roles  within  the  Navy  as  part  of  their  professional  job  within 


48 


the  formal  cyber  workforce  or  industry.  The  other  participants  primarily  engaged  in  the 
practice  as  a  hobby.  Regardless  of  this  lack  of  training  and  education,  over  half  of  the 
participants  claimed  to  be  fairly  skilled  practitioners.  The  majority  of  participants 
reported  having  experience  in  common  languages  and  repositories  used  by  programmers 
and  developers.  This  was  the  first  hackathon  that  most  participants  had  attended,  and  all 
had  similar  expectations  going  into  the  event  to  include:  expand  their  social  networks, 
improve  personal  coding  skills,  collaboration  possibilities,  and  positively  shape  and 
improve  Navy  processes. 

b.  Research  Interview  Questionnaire  ( Military  Specific) 

The  military  research  interview  questionnaire  consisted  of  fifteen  questions.  Nine 
participants  completed  this  survey,  of  which  two  participants  performed  in  a  programmer 
or  developer  related  job  within  the  Navy’s  formal  cyber  workforce.  The  remaining  seven 
participants  engaged  in  the  practice  as  a  hobby.  The  intent  of  this  survey  was  to  gather 
information  related  to  programmer  and  developer  skills  utilized  within  the  Navy  to  gain 
insights  on  how  the  Navy  might  benefit  from  a  CoP.  Below  are  the  summarized  results 
from  the  military  specific  research  interview  questionnaire. 

Most  participants  believed  their  specific  skills  were  utilized  in  their  current  Navy 
job,  but  failed  to  identify  whether  or  not  these  skills  were  related  to  the  practice  of  a 
programmer  and  developer.  Regardless,  the  majority  of  participants  thought  it  was 
necessary  for  the  Navy  to  invest  and  support  the  development  of  active-duty  military 
programmers  and  developers.  Furthermore,  they  believed  that  the  focus  of  a  CoP  should 
be  on  Navy  problems  related  to  cyberspace.  In  addition  to  the  necessity  for  Navy 
programmers  and  developers,  participants  did  not  believe  the  Navy  should  eliminate 
Navy  civilian  and  contractor  programmers  and  developers.  Participants  expressed  a  need 
for  all  three  groups  of  individuals,  because  of  the  different  levels  of  expertise  and 
experience  they  provide  which  are  necessary  for  solving  Navy  problems. 

The  majority  of  participants  suggested  that  all  officer  and  enlisted  personnel 
throughout  the  Navy  involved  with  cyberspace  operations  should  have  some  basic 
programming  skills,  which  would  allow  individuals  to  become  more  efficient  in  their 
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profession.  Since  computers  are  an  essential  part  of  maintaining  critical  DOD 
infrastructure,  all  service  members  should  learn  to  take  advantage  of  a  computer’s  power. 
All  participants  agreed  on  the  benefits  gained  from  social  and  professional  networking, 
along  with  collaboration. 

Of  the  participants,  few  active  duty  personnel  were  currently  serving  in  an  official 
programmer  or  developer  billet  or  role,  increasing  the  criticality  to  expand  social 
networks  in  order  to  improve  the  skills  of  our  active  duty  personnel.  As  one  participant 
stated: 

In  general  most  problems  are  too  complex  for  a  single  individual  to 
accomplish,  therefore  they  may  require  the  assistance  of  other  individuals 
to  tackle  the  problem.  As  systems  become  more  complex  this  will  require 
a  certain  level  of  specialization  that  may  prevent  an  individual  from 
optimally  solving  all  problems,  therefore  the  collaboration  and  networking 
may  be  of  value  to  that  individual  who  lacks  all  the  knowledge  to 
effectively  solve  the  problem.  [Anonymous] 

One  of  the  biggest  dilemmas  that  participants  felt  the  Navy  faced  when  retaining 
talented  programmers  and  developers  in  the  Navy  was  the  pay  scale  differences  between 
the  military  and  industry.  Participants  agreed  that  financial  incentives  should  be 
considered  for  military  participants  who  have  needed  programmer  and  developer  talent. 
There  were  different  thoughts  on  how  to  evaluate  the  skill  levels  of  programmers  and 
developers. 

Lastly,  participants  provided  constructive  feedback  regarding  the  location  of 
future  CoP  events.  Recommendations  were  made  to  host  more  events  in  Navy  or  military 
concentration  areas  to  minimize  travel  costs.  In  addition  to  location  considerations, 
participants’  recommend  the  Navy  provide  funding  for  both  training  and  travel  to  related 
events  and  conferences. 

c.  Research  Interview  Questionnaire  ( Civilian  Specific) 

The  civilian  research  interview  questionnaire  consisted  of  fifteen  questions.  Four 

participants  completed  this  survey.  Of  the  four  participants,  50%  were  from  the  formal 

cyber  workforce  and  the  other  50%  were  from  industry.  The  intent  of  this  survey  was  to 

gather  information  about  how  individuals  keep  their  skills  proficient,  if  prior  military 
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whether  or  not  they  utilized  their  programmer  or  developer  skills  while  in  the  military, 
personal  experience  with  CoPs,  and  their  thoughts  on  a  Navy  CoP  for  programmers  and 
developers.  Below  are  the  summarized  results  from  the  civilian  specific  research 
interview  questionnaire. 

Of  the  four  participants  from  industry  and  academia,  75%  had  extensive  military 
backgrounds,  and  all  provided  valuable  feedback.  All  participants  provided  a  spectrum  of 
insights,  despite  their  self-acclaimed  lack  of  programming  and  development  skills  and 
expertise.  All  participants  agreed  that  it  is  very  important  for  programmers  and 
developers  to  continuously  collaborate,  as  the  benefits  gained  may  help  break  down 
barriers  or  inhibitors  that  exist  between  the  U.S.  government  and  industry.  According  to 
one  participant,  two  of  the  largest  barriers  to  establishing  a  Navy  CoP  for  programmers 
and  developers  were  the  DOD  acquisition  processes  and  issues  related  to  contractors  and 
proprietary  resources.  They  believed  that  the  responsibilities,  roles,  and  restrictions  as 
outlined  in  the  DOD  acquisition  process  may  blur  the  lines  of  how  employees  could 
contribute  and  interact  within  a  Navy  CoP.  Therefore,  they  believed  it  would  be  helpful 
for  the  Navy  to  set  boundaries  on  what  the  purpose,  intent,  and  mission  of  a  Navy  CoP 
for  programmers  and  developers  should  be. 

While  participants  gave  mixed  views  on  their  potential  participation  in  a  Navy 
CoP  for  programmers  and  developers,  they  all  agreed  it  is  necessary  for  the  Navy  to  have 
skilled  programmers  and  developers,  and  that  the  Navy  could  benefit  from  this  type  of 
CoP.  A  concern  shared  by  a  participant  was  the  fact  that  Sailors  are  transient,  working  at 
any  given  command  at  any  given  time,  which  can  lead  to  duplicated  or  lost  work  across 
organizations.  Therefore,  consideration  must  be  given  to  enable  dialogue  and  interactions 
among  globally  dispersed  individuals. 

d.  Post-hackathon  Survey 

The  post-hackathon  survey  consisted  of  fourteen  questions.  Thirteen  participants 
completed  this  survey.  Some  participants  did  not  complete  this  survey  due  to  not 
attending  the  entire  event,  simply  forgot  to  complete  it,  or  felt  they  did  not  have  any 
valuable  feedback  to  contribute.  Of  the  thirteen  participants  who  did  complete  the  survey, 
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23%  performed  in  programmer  or  developer  jobs,  whereas  the  other  77%  engaged  in  the 
practice  as  a  hobby.  Furthermore  of  those  23%  who  served  in  programmer  and  developer 
related  jobs,  two  were  from  the  formal  cyber  workforce  and  one  individual  was  from 
industry.  The  intent  of  this  survey  was  to  gather  feedback  and  recommendations  on 
participants’  experience  at  HACKtheSKY.  What  follows  are  the  summarized  results  from 
the  post-hackathon  survey. 

Overall,  HACKtheSKY  exceeded  the  expectations  of  participants  with  resounding 
support  for  future  hackathons  and  social  networking  events.  Participants  believed  that 
face-to-face  interactions  were  vital  to  the  development  of  a  CoP  for  programmers  and 
developers  due  to  claimed  benefits  of  a  CoP  as  highlighted  in  Chapter  II.  Word  of  mouth 
and  social  media  were  reported  as  the  two  most  popular  means  for  how  participants  heard 
about  the  event.  Participants  left  the  three-day  event  with  shared  knowledge,  expanded 
social  and  professional  networks,  trust  with  other  participants,  and  a  desire  to  continue 
supporting  the  Navy  and  its  various  missions!  Furthermore,  participants  highly  supported 
a  combination  of  quarterly  and  semi-annual  social  interaction  events.  Some  participants 
recommended  utilizing  the  framework  of  existing  successful  communities  explored  in 
Chapter  III,  to  include  the  Athena  Project,  MMOWGLI,  and  MCPA. 

In  summary,  the  looking  phase  of  the  LUMA  Institute  human-centered  design 
approach  allowed  a  deepened  empathy  for  the  undertaking  of  programmers  and 
developers  in  order  to  gather  subjective  and  objective  feedback  used  to  better  understand 
their  needs.  Through  simplified  ethnographic  and  evaluative  research  techniques, 
participants  were  observed  in  the  act  of  programming  and  developing  cyber-related 
software,  which  allowed  better  understanding  of  user  needs.  Additionally,  interactive 
non-attributional  discussions  provided  constructive  feedback,  which  proved  important  for 
further  analysis,  because  “critical  feedback  is  more  effective  when  it  is  audible,  credible, 
and  actionable”  [78].  The  results  from  interviews,  observations,  and  surveys  will  be 
further  analyzed  in  the  understanding  phase  in  order  to  create  recommendations  for  how 
a  Navy  CoP  for  programmers  and  developers  might  be  designed  based  on  user  needs. 
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B. 


PHASE  II— UNDERSTANDING 


The  understanding  phase  focuses  on  methods  for  analyzing  information  obtained 
from  the  looking  phase  in  order  to  create  actionable  recommendations  [72].  Results 
obtained  from  interviews,  observations,  and  surveys  conducted  during  the  looking  phase 
were  further  assessed  in  order  to  determine  CoP  design  priorities  based  on  user  needs. 
Utilizing  the  People  and  Systems  approach  outlined  in  the  LUMA  method  for  the 
understanding  phase,  a  Stakeholder  Map  was  developed  in  order  to  identify  potential 
sponsorship  of  a  Navy  CoP  for  programmers  and  developers  [72]. 

1.  Looking  Phase  Analysis 

HACKtheSKY  participants  expressed  the  desire  for  NPS  to  continue  hosting 
future  hackathons  to  enable  collaboration  toward  solutions  to  military  cybersecurity 
issues.  Participants  provided  candid  and  honest  feedback  that  was  valuable  in  developing 
recommendations  for  how  the  Navy  could  best  design  a  Navy  CoP  for  programmers  and 
developers  that  is  of  value  to  users.  The  following  is  a  list  of  assessments  made  based  on 
interviews,  observations,  and  surveys  conducted  during  the  looking  phase: 

•  Through  observations  and  comments  made  among  participants,  it  is 
assessed  that  there  is  overwhelming  support  for  the  establishment  of  a 
Navy  CoP  for  programmers  and  developers.  Important  takeaways  from 
these  observations  include  the  need  to  communicate  intent,  empower 
people  at  the  edges,  and  blend  organizational  doctrine  and  technology.  It  is 
further  assessed  that  the  Navy  as  a  community  needs  to  think  in  a  different 
way  and  extend  thoughts  beyond  normal  traditions.  Communities  tend  to 
grow  more  exponentially  when  individuals  know  why  they  are  doing  what 
they  are  doing,  when  there  is  increased  emphasis  on  collaboration,  and 
creative  thinking  is  encouraged  and  accepted. 

•  There  is  a  need  for  continued  social  interactions  in  order  to  build  trust 
between  members  of  a  Navy  CoP  for  programmers  and  developers. 
Interdependence  and  trust  is  built  over  time  through  collaboration  and 
cooperation.  Creativity  and  innovative  thinking  during  events  should  be 
encouraged  among  programmers  and  developers.  Additionally,  it  is 
important  to  highlight  that  failure  is  acceptable,  and  that  we  learn  from  our 
failures  in  order  to  formulate  other  creative  ideas  to  produce  better 
products  and  solutions. 
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•  Despite  a  lack  of  formal  training  and  education  in  programming  and 
developing  participants  were  well-educated  and  had  a  great  depth  of 
experience  primarily  due  to  being  self-taught  and  motivated  to  excel  in 
their  practice.  It  is  assessed  that  the  Navy  needs  a  guild  for  proper 
development  of  talent. 

•  Social  interaction  logistics  should  include  the  consideration  of  locations 
that  encourage  maximum  participation  from  both  military  and  civilian 
participants. 

•  Collaboration  is  key  to  increasing  work  productivity  and  mission 
accomplishment.  There  are  perceived  barriers  that  exists  between  the 
government  and  the  technology  industry  that  need  to  be  addressed  and 
taken  into  consideration  when  designing  a  CoP.  Efforts  should  focus  on 
making  a  CoP  as  inclusive  as  possible,  in  order  to  encourage  the  free  flow 
of  ideas  from  a  diverse  spectrum  of  individuals  globally. 

•  Social  interaction  events  provide  value  to  both  participants  and  the  Navy, 
and  depending  on  frequency  can  continue  fostering  community  excitement 
and  involvement.  Continued  social  interactions  would  continue  the  growth 
of  a  shared  repertoire  of  tools  and  knowledge,  allowing  members  to 
further  collaborate  in  order  to  take  purposeful  action  towards 
accomplishing  mission  requirements.  These  resources  may  include  tools, 
methods,  lessons  learned,  databases,  websites,  or  the  use  of  anything  that 
improves  and/or  provides  content  or  relevancy  to  the  practice  of  a  CoP. 
Additionally,  these  projects  may  be  in  the  form  of  meetings,  conferences, 
or  team  building  events  such  as  formalized  community  challenges. 

In  summary,  results  from  the  looking  phase  provided  constructive  feedback  for 
further  analysis  and  assessments  to  be  made  in  regards  to  how  best  design  a  Navy  CoP 
for  programmers  and  developers  with  the  focus  centered  on  the  needs  of  users.  In  order  to 
continue  the  design  of  a  Navy  CoP,  potential  sponsorship  must  be  identified  in  order  to 
further  create  and  support  the  establishment  of  a  Navy  CoP  for  programmers  and 
developers. 

2.  People  and  Systems 

The  developed  understanding  and  empathy  gained  in  the  looking  phase  can  lead  to 

forms  of  innovation  that  can  utilize  this  new  knowledge  in  order  to  visually  transcend  and 

reinvent  it.  The  Peoples  and  Systems  approach  is  an  array  of  techniques  and  tools  the 

“support  the  ability  to  consider  all  kinds  of  people,  places,  and  things  in  pursuit  of 

creating  a  new  value”  [72].  One  recommended  technique  that  is  of  value  in  creating  a 
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Navy  CoP  for  programmers  and  developers  is  the  creation  of  a  Stakeholder  Map,  which 
allows  individuals  to  visualize  potential  sponsoring  organizations  who  might  have  a 
“vetted  interest  in  the  success  and  outcomes  of  a  community”  [72].  A  Stakeholder  Map 
includes  potential  sponsors,  champions,  and  stakeholders  who  have  interest  in  the 
practice  of  the  intended  audience  and  the  outcomes  that  result  from  their  profession.  The 
creation  of  a  Stakeholder  Map  should  be  intertwined  with  a  team  effort  concept  vice  a 
one-sided  design  in  order  to  observer  all  ideas  and  suggestions.  The  idea  behind  this 
technique  is  to  “include  a  very  broad  range  of  sponsors,  champions,  and  stakeholders,  and 
strike  a  balance  between  breadth  and  relevance”  [72].  The  benefits  from  this  technique 
include  the  identification  and  potential  recruitment  of  sponsorship  needed  for  support  and 
the  launch  of  a  CoP  in  order  to  help  guide  plans  for  future  prototyping,  growth,  and 
sustainment. 

At  this  moment  a  Navy  CoP  for  programmers  and  developers  is  an  exceptional 
idea,  but  it  needs  the  support  from  higher  echelon  commands  to  move  forward.  The 
author  recommends  that  the  Deputy  Chief  of  Naval  Operations  for  Information 
Dominance/Director  of  Naval  Intelligence  (OPNAV  N2/N6)  be  the  CoP  sponsor,  with 
support  from  subordinate  commands.  Establishing  their  presence  and  commitment  as  a 
sponsor  will  provide  credibility  and  legitimacy  to  the  continued  promotion  of  a  Navy 
CoP  for  programmers  and  developers.  Once  this  relationship  is  established,  critical  items 
of  interest  such  as  mission  and  vision  statements  can  start  to  be  created. 

As  OPNAV  N2/N6,  they  are  responsible  for  the  Information  Warfare  Community, 
and  have  a  vested  interest  in  utilizing  the  power  of  a  Navy  CoP  for  programmers  and 
developers  due  to  its  strategic  objectives  [3].  In  order  to  grow  the  formal  and  informal 
cyber  workforces,  there  must  be  buy-in  from  the  top  down.  This  top-level  support  from 
OPNAV  N/2/N6  may  help  seek  out  the  most  qualified  personnel  that  currently  have  the 
talent,  but  no  interest  in  joining  the  formal  cyber  workforce.  Once  these  individuals  are 
located,  they  can  be  introduced  to  the  formal  cyber  community  and  asked  to  spread  the 
word  that  the  Navy  is  seeking  to  expand  its  cyber  workforces.  Additionally,  the  Navy  has 
a  great  overall  recruiting  theme  to  maintain  its  manning  levels,  but  such  a  campaign  can 
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be  utilized  to  create  a  much  larger  CoP  by  collaborating  with  the  youth  that  will  be  our 
future. 

As  the  sponsor,  OPNAV  N2/N6  would  provide  a  Navy  CoP  with  the  legitimacy 
and  gravitas  needed  to  expand  the  reach  of  the  community  as  was  demonstrated  in 
successful  communities  evaluated  in  Chapter  III.  OPNAV  N2/N6  also  has  enough 
influence  and  authority  to  offer  credibility  for  CoP  events.  OPNAV  N2/N6  would  be 
responsible  for  promoting  the  adoption  of  a  Navy  CoP  for  programmers  and  developers 
to  senior  ranking  officials  in  order  to  recruit  future  sponsors  and  stakeholders  such  as  the 
Chief  of  Naval  Operations  and  Navy  Type  Commanders.  While  this  support  is  not 
expected  to  be  immediate,  opening  a  continuous  dialogue  can  start  the  process  of 
designing  and  establishing  a  sustainable  CoP  for  the  long-term. 

Tenth  Fleet  is  the  recommended  champion  for  a  Navy  CoP  for  programmers  and 
developers.  Tenth  Fleet  carries  credibility  and  legitimacy  in  their  name  and  reputation 
throughout  the  military,  academia,  and  industry  communities.  Tenth  Fleet  has  the  ability 
to  channel  into  other  networks  in  the  military  and  civilian  sectors.  These  channels  may 
provide  manpower  or  monetary  support  for  future  events,  and  the  establishment  of  a 
Navy  CoP  for  programmers  and  developers. 

Figure  4  shows  that  the  recommended  stakeholders  for  a  Navy  CoP  for 
programmers  and  developers  should  include  the  U.S.  Navy  system  commands 
(SYSCOMs),  Joint  Task  Force  -  DOD  Information  Networks  (JTF-DODIN),  and  Tenth 
Fleet  standing  forces.  The  SYSCOMs  make  up  the  informal  cyber  workforce  commands 
and  are  responsible  for  the  “design,  construction,  and  maintenance  of  military  systems 
including  ships,  aircraft  and  weapons”  [79].  The  SYSCOMs  recommended  as 
stakeholders  include  Naval  Air  Systems  Command  (NAVAIR),  Naval  Sea  Systems 
Command  (NAVSEA),  Space  and  Naval  Warfare  Systems  Command  (SPAWAR),  Naval 
Facilities  Engineering  Command  (NAVFAC),  Marine  Corps  Systems  Command 
(MCSC),  and  the  Office  of  Naval  Research  (ONR)  [79]. 
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Figure  4.  Recommended  Stakeholders  for  a  Navy  CoP 


The  Tenth  Fleet  standing  forces  that  make  up  the  formal  cyber  workforce  and  are 
recommended  as  initial  stakeholders  include  all  Commander  Task  Forces  (CTF) 
associated  with  Network  Operations  and  Defense,  10,  Service  Cryptologic  Component 
Operations,  Fleet  and  Theatre  Operations,  and  Research  and  Development,  as  well  as 
Naval  Information  Forces  (NAVIFOR).  CTFs  are  identified  by  a  four-digit  number  and 
command  location.  The  following  CTFs  are  included  as  recommended  stakeholders: 
Fleet  Cyber  Command/Tenth  Fleet,  CTF  1000  (C10F),  CTF  1010  (Naval  Network 
Warfare  Command),  CTF  1020  (NCDOC),  CTF  1030  (NIOC  Norfolk),  CTF  1040  (NIOC 
Texas),  CTF  1050  (NIOC  Georgia),  CTF  1060  (NIOC  Maryland),  CTF  1070  (NIOC 
Hawaii),  CTF  1080  (NIOC  Colorado),  and  CTF  1090  (NCDWG). 

These  recommended  stakeholders  represent  the  respective  forces  and 
organizations  that  may  help  strengthen  a  Navy  CoP  for  programmers  and  developers. 
Stakeholders  should  be  focused  on  the  general  populace  to  include,  civilian  and  DOD.  A 
CoP  would  allow  the  Navy  to  utilize  external  experience  and  knowledge  in  latest 
technology  and  practices,  which  can  ultimately  improve  Navy’s  current  cyber  operations. 
The  civilian  workforce  operates  under  a  different  set  of  rules  and  regulations,  which  at 
times  may  provide  its  members  additional  flexibility  to  operate  outside  of  normal 
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boundaries  of  Navy  constraints.  A  CoP  could  assist  industry  in  better  understanding  Navy 
restraints  and  constraints  in  order  to  develop  tools  and  procedures  to  better  assists  in 
protecting  key  infrastructures.  Collaboration  and  team  building  between  formal  and 
informal  cyber  workforces  and  industry  equivalent  may  expand  the  Navy’s  cyber 
footprint  globally. 

All  recommended  sponsors,  champions,  and  stakeholders  may  benefit  from  being 
involved  in  a  Navy  CoP  for  programmers  and  developers.  Ideally,  they  should  all  have  an 
interest  in  the  accomplishments  and  goals  of  the  CoP.  All  recommended  sponsoring 
organizations  (sponsor,  champion,  and  stakeholders)  represent  personnel  and 
organizations  both  within  the  Navy’s  formal  and  informal  cyber  workforces.  This 
dynamic  group  of  organizations  may  help  in  building  relationships  and  bridges  between 
these  formal  and  informal  workforces  as  well  as  with  industry. 

In  summary,  results  of  the  looking  phase  interviews,  observations,  and  surveys 
contributed  to  further  assessments  and  the  identification  of  potential  sponsoring 
organizations  in  the  understanding  phase.  The  assessments  and  identifications  examined 
in  the  understanding  phase  will  be  used  in  the  last  phase  of  the  LUMA  process,  the 
making  phase,  in  order  to  create  recommendations  for  how  to  best  design  a  Navy  CoP  for 
programmers  and  developers.  Assessments  bolster  the  support  of  the  proposed  Navy  CoP 
due  to  the  benefits  to  be  gained  as  highlighted  in  Chapters  II  and  III.  Furthermore,  with 
the  identification  of  potential  sponsoring  organizations,  focus  can  shift  to  the  making 
phase  in  order  to  provide  relevant  design  recommendations  for  a  Navy  CoP  for 
programmers  and  developers. 

C.  PHASE  III— MAKING 

The  making  phase  focuses  on  methods  and  approaches  that  can  be  used  to 
effectively  design  a  CoP  for  Navy  programmers  and  developers.  Through  a  series  of 
brainstorming  techniques,  calculated  risks  in  various  methods  of  prototypes  and 
innovation,  and  design  rationale,  the  author  created  a  generalized  checklist  that  may  be 
used  to  design  any  CoP.  The  checklist  is  broken  down  into  the  following  categories  as 
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defined  by  common  key  characteristics  described  in  Chapter  III:  Social  Interactions, 
Structure,  and  Lifecycle.  Furthermore,  Structure  is  broken  down  into  the  following  sub 
categories  as  defined  in  Chapter  III:  1)  Sponsors,  Champions,  and  Stakeholders,  2) 
Mission,  Vision,  and  Purpose,  3)  Membership  Framework,  and  4)  Measures  of  Success. 

1.  General  Guidelines 


□ 

CoP  establishment  should  not  be  mandated.  Use  a  “light  hand”  when  creating  the 
CoP.  Otherwise  members  may  be  discouraged  to  join  and  participate. 

□ 

The  CoP  should  remain  simple  and  informal.  If  too  many  demands  or 
expectations  are  imposed  on  CoP,  then  the  focus  shifts  to  satisfying  the  bosses’ 
tasks,  and  further  discourages  CoP  membership  and  participation. 

□ 

During  the  design  phase  of  a  CoP,  the  focus  at  all  times  should  remain  on  the 
purpose  for  which  the  CoP  is  being  created  and  designed.  The  primary  focus 
should  be  for  individuals  to  learn  from  each  other  as  a  result  of  shared 
knowledge  and  collaboration,  while  producing  tangible  results. 

□ 

Start  small  and  evolve  when  designing  a  CoP.  There  is  no  rush  to  designing  a 
CoP.  Allow  for  the  time  to  develop  and  prototype.  The  real  useful  action  of  a 
CoP  is  the  interactions  between  members,  not  necessarily  the  technological 
means  they  have  available. 

□ 

Remember  that  the  idea  of  continuous  CoP  evaluation  is  that  it  offers  the  CoP 
team  an  opportunity  to  learn  and  reflect,  with  the  understanding  that  it  takes 
days,  months,  and  years  to  design  a  successful  CoP.  Additionally,  continuous 
assessment  and  re-evaluation  on  the  return  on  investment  of  participation  is  key 
for  determining  commitment. 

2.  Social  Interactions 


□ 

Are  CoP  champions  assigned  and  designated  in  order  to  coordinate  and  plan 
social  interaction? 

□ 

What  is  the  annual  projected  social  interaction  plan? 

□ 

Is  at  least  one  of  the  annual  social  interactions  a  face-to-face  interaction?  If  no, 
plan  for  at  least  one  face-to-face  interaction  if  possible,  feasible,  and  affordable. 

□ 

Are  social  interaction  problem  statements  and  projects  planned  in  order  to 
produce  tangible  results? 
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□ 

Are  sponsors  willing  and  able  to  host  regional  meetings  and  conferences  in  order 
to  encourage  the  improvement  and  mentoring  of  member’s  practice? 

□ 

Have  CoP  members  been  informed  on  local  community  events  that  they  may 
attend  in  order  to  seek  mentors  internal  and  external  to  their  organization? 

□ 

What  team-building  events  have  been  planned  to  include  community  challenges 
and  encourage  max  participation? 

□ 

Are  social  interactions  open  to  the  public? 

□ 

Is  there  an  established  CoP  website? 

□ 

Is  the  CoP  website  used  to  promote  annual  social  interactions  and  events? 

□ 

Does  the  CoP  website  provide  CoP  members  a  forum  to  collaborate,  network, 
and  share  knowledge,  tools,  and  experiences? 

□ 

Are  established  after  action  reports  and/or  de-briefs  planned  post  social 
interactions?  Who  has  been  assigned  this  task? 

3.  Structure 

a.  Sponsors,  Champions,  and  Stakeholders 


□ 

Who  are  the  recommended  Sponsors?  Note:  May  only  have  one  Sponsor. 

□ 

Who  are  the  recommended  Stakeholders? 

□ 

Who  are  the  recommended  Champions? 

□ 

Do  all  sponsors,  champions,  and  stakeholders  respectively  provide  credibility, 
legitimacy  to  the  CoP? 

□ 

Do  sponsoring  organizations  support  and  have  a  vetted  interest  in  the  CoP? 

□ 

Do  sponsoring  organizations  know  how  they  will  benefit  from  and  contribute  to 
the  CoP? 
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□ 

Are  sponsoring  organizations  willing  to  remove  barriers  and  obstacles  that  may 
prevent  CoP  productivity? 

□ 

What  previous  experience  do  sponsoring  organizations  have  with  current  or 
existing  CoPs?  Do  they  understand  their  current  roles  and  responsibilities  within 
the  CoP? 

□ 

How  should  the  sponsoring  organizations  interact  with  CoP  members,  and  how 
frequently  should  these  interactions  occur? 

□ 

Are  sponsoring  organizations  capable  of  providing  the  necessary  infrastructure 
and/or  support  as  needed  by  CoP? 

b.  Mission ,  Vision,  and  Purpose 


□ 

Are  mission  and  vision  statements  short  and  concise? 

□ 

Are  mission  and  vision  statements  publically  displayed  for  all  current  and  future 
members  to  view? 

□ 

Did  sponsoring  organizations  provide  input  in  regards  to  CoP  mission,  vision, 
and  purpose? 

□ 

Do  the  mission  and  vision  statements  align  with  the  goals  of  sponsoring 
organizations  and  their  key  issues? 

□ 

Does  the  CoP  have  focused  and  well-defined  purposes?  Where  can  these 
purposes  be  found? 

□ 

Are  the  purposes  publically  displayed  for  all  current  and  future  members  to 
view? 

□ 

Do  the  purposes  reflect  and  include  learning  through  shared  knowledge, 
collaboration,  and  the  acts  of  taking  purposeful  action  in  order  to  successfully 
execute  tasks  and  projects? 

□ 

Do  the  purposes  highlight  member  benefits  gained  from  active  CoP 
participation? 

c.  Membership  Framework 


□ 

Who  is  the  intended  audience  for  the  CoP? 

□ 

Is  CoP  membership  inclusive? 
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□ 

Is  CoP  membership  volunteer  based? 

□ 

Does  the  CoP  website  include  a  list  of  potential  benefits  gained  from  being  an 
active  member  within  the  CoP? 

□ 

Does  the  CoP  website  inform  current  and  future  CoP  members  suggestions  on 
how  they  may  contribute  or  add  value  to  the  CoP? 

□ 

What  costs  might  be  incurred  by  the  CoP  in  order  to  afford  the  opportunity  for 
CoP  members  to  effectively  social  network  and  collaborate? 

□ 

What  is  the  current  assessment  and  re-evaluation  plan  for  determining  the  return 
on  investment  of  participation  in  a  CoP? 

□ 

How  is  membership  commitment  to  the  CoP  determined? 

d.  Measures  of  Success 


□ 

Have  sponsoring  organizations  provided  input  in  regards  to  what  they  think  a 
successful  CoP  may  look  like? 

□ 

Is  there  an  established  guideline  or  checklist  for  post-event  assessment  in  order 
to  measure  the  success  of  an  event? 

□ 

What  is  the  plan  of  action  for  assessing  how  the  CoP  is  progressing? 

□ 

How  might  we  assess  the  level  of  trust  earned  and  developed  between  and 
among  CoP  members? 

□ 

Have  pre  and  post  surveys  been  created  for  events  in  order  to  evaluate  events  and 
received  candid  feedback,  suggestions,  and  recommendations  for  how  to 
improve  future  events?  Are  these  surveys  non-attributional? 

□ 

What  metrics  are  best  for  measuring  success  of  the  CoP? 

4.  Lifecycle 


□ 

How  often  will  the  lifecycle  of  the  CoP  be  assessed? 

□ 

Are  interested  sponsoring  organizations  involved  in  lifecycle  assessment 
planning  meetings? 

□ 

What  is  the  plan  for  post  social  interaction  assessment  meetings  in  order  to 
evaluate  event  feedback,  number  of  members,  frequency  among  member 
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collaboration,  and  future  planning? 

□ 

What  available  means  are  there  for  evaluating  the  lifecycle  of  the  CoP?  Note: 
Teleconferences  may  accommodate  a  larger  audience  in  order  to  receive  more 
input  from  sponsoring  organizations. 

□ 

What  are  the  levels  of  energy,  commitment,  and  visibility  within  the  CoP? 

□ 

Is  it  recommended  that  a  certain  phase  be  repeated  in  order  to  potentially  re¬ 
design,  prototype,  or  launch  a  new  idea? 

This  checklist  is  one  recommended  tool  for  how  a  Navy  CoP  for  programmers 
and  developers  may  be  designed.  This  checklist  was  created  with  specific  focus  on  key 
characteristics  within  successful  CoPs,  and  is  therefore  not  inclusive  of  all  possible  CoP 
characteristics.  As  with  the  LUMA  Institute  human-centered  design  approach  and 
handbook,  any  step  throughout  this  checklist  may  be  skipped  if  it  is  not  applicable  to  the 
design  the  CoP. 

D.  CONCLUSION 

The  LUMA  Institute  human-centered  design  approach  provides  a  framework  for 
incorporating  and  analyzing  feedback,  suggestions,  and  recommendations  on  how  to 
design  a  Navy  CoP.  Results  from  the  looking  phase  collected  from  a  diverse  collection  of 
military  and  civilian  participants  at  HACKtheSKY  provided  insight  into  an  expressed 
desire  and  need  for  a  Navy  CoP  for  programmers  and  developers.  Analysis  of  these 
insights  in  the  LUMA  understanding  phase  shed  light  on  specific  points  of  concern  when 
designing  a  CoP,  to  include  potential  membership,  levels  of  contribution,  benefits,  and 
recommendations  for  future  CoP  social  interactions.  Finally,  in  the  making  phase  these 
considerations  and  key  characteristics  were  coalesced  to  develop  a  specific  set  of  check 
points  in  order  to  launch  a  CoP  tailored  to  user  needs  and  desires. 

Chapter  VI  ties  in  supporting  evidence  from  previous  chapters  and  insights  gained 
from  implementation  of  the  LUMA  Institute  human-centered  design  handbook  in  order  to 
provide  specific  actions  recommended  for  Tenth  Fleet,  the  proposed  champion  and  chief 
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organizer  of  events,  to  initiate  the  establishment  of  and  future  launch  of  a  Navy  CoP  for 
programmers  and  developers. 
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VI.  CONCLUSION  AND  RECOMMENDATIONS 


The  main  research  objective  of  this  work  was  to  explore  how  the  Navy  may 
benefit  from  a  CoP  for  programmers  and  developers. 

The  Navy  stands  to  benefit  from  the  creation  of  a  CoP  for  programmers  and 
developers  sponsored  by  the  formal  cyber  workforce  because  of  the  tangible  benefits 
gained.  Overall  CoP  benefits  include  expanded  social  boundaries,  shared  knowledge,  and 
a  shared  repertoire  of  tools  and  resources  that  may  be  used  to  improve  how  organizations 
execute  and  accomplish  missions.  Expanded  social  boundaries  enable  dialogue  between  a 
diverse  global  spectrum  of  individuals,  which  may  unlock  innovative  creativity,  and 
through  collaboration  capture  and  diffuse  knowledge  in  order  to  deliver  tangible  results. 

These  benefits  are  often  due  to  the  structural  ways  that  CoPs  invite  and  merge 
diverse  groups  of  individuals  together  in  order  to  collaborate  on  solving  a  problem.  The 
establishment  of  a  Navy  CoP  for  programmers  and  developers  provides  bi-directional 
tangible  benefits  to  the  formal  and  informal  cyber  workforces  and  industry.  Benefits 
include  global  access  to  all  levels  of  expertise,  mentorship,  connectedness,  increased 
cybersecurity  of  computer  dependent  systems,  exposure  to  the  cybersecurity  needs  and 
requirements  of  the  Navy,  and  exposure  to  the  latest  and  greatest  technology,  tools,  and 
techniques  found  in  industry. 

Furthermore,  a  Navy  CoP  for  programmers  and  developers  should  have  the 
features,  tools,  or  characteristics  of  GitHub,  Stack  Overflow,  and  Top  Coder.  These 
features  include:  a  global  community  organized  around  a  certain  domain  identity  that 
advocates  developers  act  as  change  agents;  incentives  for  interacting,  mentoring,  and 
coaching;  and  competitions  that  provide  incentives  that  encourage  collaboration  to 
produce  tangible  results.  These  benefits  are  often  attributed  to  the  structural  elements  of  a 
CoP.  These  elements  are  defined  by  key  characteristics  shared  among  successful  CoPs. 

Key  characteristics  of  successful  CoPs  should  be  taken  into  consideration  when 
exploring  design  approaches  and  techniques  for  creating  a  CoP.  Various  forms  of  social 
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interactions  are  important  to  building  relationships  and  developing  trust  among  members 
in  order  to  encourage  continued  active  participation  within  the  CoP.  Social  interactions 
would  only  promote  and  maintain  the  stamina  of  a  new  CoP  for  so  long  until  the  CoP 
needed  credible  and  legitimate  support  from  sponsoring  organizations.  Therefore  it  is 
important  to  identify  and  establish  sponsors,  champions,  and  stakeholders  that  have  a 
vetted  interested  in  the  success  and  outcomes  of  the  CoP  early  on  in  the  creation  of  the 
community.  Once  sponsoring  organizations  are  established,  they  can  assist  in  the 
development  of  the  mission,  vision,  and  purpose  of  the  CoP.  These  statements  should  be 
accessible  to  all  interested  parties  in  order  to  recruit  and  promote  an  inclusive 
membership  framework,  that  is  centered  on  shared  identifies  of  members’  practice.  This 
sense  of  identity  may  foster  continued  commitment  to  the  success  of  the  CoP,  and  may 
also  be  used  to  measure  success. 

Furthermore,  a  Navy  CoP  for  programmers  and  developers  should  have  the  shared 
characteristics  of  existing  successful  communities  to  include:  Athena,  Black  Hat,  Def 
Con,  APAN,  MMOWGLI,  and  Company  Command.  These  characteristics  include:  small 
and  large  scale  face-to-face  events;  a  virtual  space  designed  and  hosted  by  DOD  that 
enables  collaboration  across  communities  within  the  Navy  and  DOD  as  well  as  between 
the  public  and  private  sectors;  inclusive  membership;  and  clear  measures  of  success.  All 
of  these  characteristics  factor  into  the  lifecycle  of  a  CoP  and  should  be  taken  into 
consideration  when  designing  a  Navy  CoP  for  programmers  and  developers. 

Design  approaches  and  techniques  often  focus  on  end  results  influenced  by  users’ 
needs  and  encourage  collaborative  thinking  among  vastly  dispersed  groups  of  individuals 
in  order  to  design  something  with  intent  and  purpose.  Furthermore,  a  human-centered 
Design  Thinking  approach  “puts  the  needs  of  users  at  the  center  of  the  design  process,” 
and  through  observations  and  collaboration  encourages  innovative  thinking  of  all 
involved  in  order  to  design  the  best  possible  product  [66].  The  LUMA  Institute  human- 
centered  design  approach  simplifies  Design  Thinking  processes  into  three  simple  phases: 
looking,  understanding,  and  making.  These  phases  are  conducted  through  interviews, 
observations,  surveys,  stakeholder  maps,  and  design  checklists  in  order  to  produce  the 
best  solution  based  on  user  needs. 
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In  conclusion,  a  Navy  CoP  for  programmers  and  developers  has  the  potential,  if 
designed  purposefully  and  given  the  time  to  mature,  to  enrich  the  body  of  knowledge  and 
practice  of  cybersecurity  and  cyber  operations  for  programming  and  developing  software 
of  consequence  to  the  service.  However,  a  successful  CoP  is  not  created  overnight.  CoPs 
take  time  to  “emerge,  to  flourish  and  to  become  productive”  [9].  It  may  take  a  series  of 
iterations  and  innovative  prototypes  to  find  what  does  and  does  not  work.  Therefore, 
time,  patience,  and  prototype  will  be  necessary  to  further  design  a  Navy  CoP  for 
programmers  and  developers  in  order  to  reap  the  claimed  benefits  of  a  CoP. 

A.  RECOMMENDATIONS  FOR  TENTH  FLEET 

The  author  recommends  Tenth  Fleet,  as  the  recommended  CoP  champion  and 
chief  organizer  of  CoP  events,  be  ultimately  responsible  for  the  supporting,  prototyping 
launching,  and  growing  of  the  proposed  Navy  CoP  for  programmers  and  developers.  As 
seen  in  Figure  5,  this  thesis  research  made  an  assumption  that  the  Navy  would  benefit 
from  the  design  of  a  Navy  CoP  for  programmers  and  developers,  it  inquired  about  the 
benefits  of  the  proposed  CoP,  proposed  design  techniques  and  approaches,  and  conducted 
minimal  prototyping.  Chapter  II  and  Chapter  III  inquired  the  benefits  and  characteristics 
of  CoPs.  Chapter  IV  discussed  design  approaches  and  techniques  to  use  when  designing  a 
CoP.  Chapter  V  implemented  the  LUMA  Institute  human-centered  design  approach  and 
tested  looking,  understanding ,  and  making  techniques  in  order  to  make  further 
recommendations  for  prototyping  and  future  design  of  a  Navy  CoP  for  programmers  and 
developers.  At  this  point,  it  is  recommended  that  Tenth  Fleet  become  responsible  for  the 
continued  prototyping  and  ultimately  launch  the  CoP  in  order  for  it  to  grow  and  sustain  as 
a  living  entity. 
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Figure  5.  Community  of  Practice  Lifecycle  Phases.  Adapted  from  [14]. 

We  believe  that  the  Navy  will  benefit  from  a  CoP  for  programmers  and 
developers.  Through  the  work  provided  here,  Tenth  Fleet  is  provided  with 
recommendations  based  on  literature  review  and  characteristics  of  existing  communities 
for  how  to  best  design  a  successful  and  sustainable  CoP  for  programmers  and  developers. 
Below  are  specific  actions  that  are  recommended  to  support,  adopt,  and  see  through  to 
completion  in  order  to  initiate  the  establishment  of  a  Navy  CoP  for  programmers  and 
developers.  These  recommendations  took  into  consideration  all  literature  review, 
examination  of  existing  successful  communities,  and  recommendations  received  through 
the  use  of  surveys,  questionnaires,  interviews,  and  observations  obtained  utilizing  the 
LUMA  Institute  human-centered  Design  Thinking  approach. 

1.  Social  Interactions 

The  Navy  formal  cyber  community  should  conduct  quarterly  social  interaction 
events.  Two  of  these  quarterly  events  should  be  face-to-face  interactions,  specifically 
semi-annual  hackathons,  given  the  importance  of  establishing  trust  and  repertoire  within 
and  among  CoP  members.  In  addition  to  semi-annual  hackathons,  we  recommend  that 
members  of  the  CoP  determine  the  other  two  quarterly  events.  Planning  and  preparation 
for  a  hackathon  is  both  time  consuming  and  costly,  therefore  the  other  two  events  should 
be  inexpensive  and  require  little  time  to  design  and  host.  Some  suggestions  received  from 
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survey  feedback  were  to  conduct  Athena  Project-related  events  in  Navy  concentration 
areas.  Given  the  diversity  and  number  of  Athena  groups  located  throughout  the  Navy, 
potential  Athena  members  could  volunteer  their  time  to  coordinate  with  sponsors  and 
stakeholders  on  hosting  a  programmer-  and  developer-related  Athena  Project  event.  This 
would  also  provide  a  platform  for  Sailors  to  pitch  their  cyber-related  ideas  that  could 
potentially  be  used  for  future  hackathons.  Additionally,  this  approach  could  potentially 
recruit  more  membership  participation  from  within  the  formal  and  informal  cyber 
community,  therefore  expanding  the  Navy’s  cyber  workforce. 

In  the  subsections  that  follow,  some  mechanisms  for  social  interactions  are 
described.  They  include  semi-annual  hackathons,  an  online  web  presence,  and 
cybersecurity  conferences. 

a.  Semi-annual  Hackathons 

HACKtheSKY  was  a  success,  and  positive  feedback  was  received  from  all 
participants,  as  well  as  the  desire  and  interest  to  continue  similar  face-to-face  interactions. 
Hackathon  locations  should  move  from  key  cities  to  key  Navy  concentration  areas,  in 
order  to  draw  maximum  participation  from  the  Navy  and  industry.  The  first  hackathon 
was  conducted  in  San  Francisco,  but  feedback  received  supported  the  following  locations 
for  future  hackathons:  Austin,  Boston,  Seattle,  San  Diego,  Hawaii,  and  Washington  DC. 

A  second  hackathon,  HACKtheMACHINE,  is  being  planned  for  Austin,  Texas  in 
February  2017.  It  is  recommended  that  a  third  hackathon  be  located  on  a  military 
installation,  preferable  a  location  that  has  large  platform  ships  and/or  aircraft.  Hosting  a 
hackathon  on  a  military  installation,  especially  Navy  installations,  can  be  motivation  for 
industry  to  further  participate  and  engage  in  the  establishment  of  a  Navy  CoP  for 
programmers  and  developers.  Though  the  hackathon,  industry  participants  may  gain  a 
deeper  understanding  of  the  needs  and  requirements  of  the  Navy. 

The  CoP  champion  should  ensure  that  all  hackathons  have  appropriate  resources. 
This  may  include  travel  and  accommodations  for  a  designated  number  of  active  duty 
military  participants.  A  Tenth  Fleet  organization  provided  such  support  to 
HACKtheSKY,  and  it  was  an  incentive  for  military  personnel  to  participate.  Sponsors 
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and  stakeholders  should  also  provide  input  for  hackathon  projects.  The  CoP  champion 
should  take  the  lead  in  polling  all  potential  stakeholders  for  project  ideas,  and  use  those 
as  a  guideline  for  how  to  advertise  and  recruit  for  future  hackathons.  Investigation  of  the 
most  effective  format  for  a  CoP  hackathon  should  be  conducted. 

b.  Online  Web  Presence 

An  online  web  presence  that  would  allow  CoP  members  to  collaborate  and  share 
knowledge  should  be  considered.  Support  for  a  web  presence  was  echoed  in  all  surveys 
received  from  HACKtheSKY  participants.  The  purpose  of  an  online  web  presence  would 
be  to  post  information  pertaining  to  future  CoP  events  and  activities,  as  well  as  provide  a 
place  for  the  free  flow  of  ideas  and  collaboration,  questions  and  answers,  and  shared 
knowledge. 

Due  to  the  inclusiveness  of  membership,  it  is  recommended  that  a  group  site  be 
established  and  created  on  APAN,  which  is  an  “unclassified  information  sharing  service 
for  the  U.S.  DoD”  [37].  The  author  could  create  the  initial  group  site  with  the  assistance 
of  HACKtheSKY  participants.  Those  participants  could  provide  feedback  and  inputs  in 
regards  to  design  and  functionality  of  the  website.  The  website  should  be  maintained  for 
at  least  one  year  in  order  to  evaluate  its  usefulness  and  purpose.  One  way  to  measure  its 
usefulness  may  be  to  survey  registered  users  and  ask  for  their  feedback.  The  website 
might  also  provide  various  metrics  for  measuring  success.  If  after  a  year,  the  website  does 
not  prove  to  serve  a  purpose  or  fails  to  be  used  by  CoP  members,  then  trying  a  different 
website  such  as  that  of  MCPA,  which  already  has  an  established  cyber  presence,  should 
be  considered.  Regardless  of  the  website  chosen,  the  primary  objective  for  its  purpose 
and  functionality  is  to  meet  and  address  the  needs  of  CoP  members. 

c.  Cybersecurity  Conferences 

Sponsors  and  stakeholders  should  encourage  interested  service  members  from  the 

formal  and  informal  cyber  workforces  to  become  actively  involved  with  local 

cybersecurity  conferences.  These  conferences  provide  benefits  such  as  social  networking, 

sharing  knowledge,  and  improving  basic  skills.  It  is  recommended  that  the  formal  cyber 

community  promote  participation  by  service  members  in  addition  to  logistic  support  to 
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cover  the  costs  of  travel  and  conference  expenses.  Policy  changes  should  be  made  that 
incorporate  attainable  goals  for  how  Sailors  can  become  more  proficient  programmers 
and  developers,  with  cybersecurity  conference  attendance  and  participation  being  one 
solution.  These  goals  should  not  be  mandated  or  required,  but  should  serve  as  tools  and 
suggestions  for  how  Sailors  may  be  more  proactive  in  becoming  proficient  at  their 
practice.  Additionally,  guidance  should  be  given  to  Sailors  on  how  these  conference  may 
benefit  them  professionally  and  personally,  as  well  as  how  they  can  mentor  and  teach 
others  who  were  not  able  to  attend  as  a  way  of  giving  back  to  the  community  and  CoP  at 
large. 


2.  Structure 

CoP  sponsors,  champions,  and  stakeholders  must  first  be  identified  and  agree  to 
assume  the  roles  and  responsibilities  of  their  title  before  other  structural  elements  of  a 
CoP  are  addressed  and  developed.  Once  sponsoring  organizations  have  been  identified, 
actions  may  then  be  taken  to  establish  mission,  vision,  purpose,  membership  framework, 
and  measures  of  success.  In  combination,  these  elements  can  create  the  foundation  for  the 
design  of  a  successful  CoP.  Chapter  VI  recommended  that  OPNAV  N2/N6  be  the  CoP’s 
core  sponsor,  Tenth  Fleet  the  CoP  champion,  and  U.S.  Navy  system  commands 
(SYSCOMs)  and  Tenth  Fleet  standing  forces  would  be  the  recommended  stakeholders 
for  a  Navy  CoP  for  programmers  and  developers.  All  recommended  sponsoring 
organizations  represent  personnel  and  organizations  both  within  the  Navy’s  formal  and 
informal  cyber  workforces,  and  should  have  an  interest  in  the  accomplishments  and  goals 
of  the  proposed  CoP.  This  dynamic  group  of  organizations  may  help  build  relationships 
and  bridges  between  these  formal  and  informal  workforces  as  well  as  with  industry. 

a.  Mission,  Vision,  and  Purpose 

Sponsoring  organizations  and  CoP  members  have  valuable  ides  and  insights  that 
may  contribute  to  determining  appropriate  mission,  vision,  and  purpose  of  a  Navy  CoP 
for  programmers  and  developers.  It  is  recommended  the  CoP  host  a  design  event  in  order 
to  crowdsource  ideas  for  simple,  but  often  time  consuming  tasks,  such  as  determining 
mission  and  vision  statements,  purposes,  benefits,  and  CoP  membership  guidelines.  A 
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design  event  could  help  promote  further  membership  by  using  the  ideas  and  solutions 
created  by  innovative  participants.  Furthermore,  the  design  event  could  also  focus  on  the 
CoP  web  presence,  and  how  to  best  design  an  active  webpage  that  meets  the  needs  and 
desires  of  CoP  members.  Tenth  Fleet  should  be  responsible  for  the  organization  and 
logistics  of  this  event.  A  MMOWGLI  call  to  action  may  be  a  way  to  crowdsource  a  large 
diverse  audience  to  generate  design  ideas. 

b.  Membership  Framework 

Membership  should  be  inclusive  and  focus  on  the  establishment  of  CoP  member 
commitment.  Providing  incentives  to  active  participating  members  often  encourages 
membership  commitment.  Two  cost  efficient  ideas  to  further  commitment  of  potential 
CoP  members  are  reputation  points  and  a  repository  of  programmer  and  developers  tools. 
These  incentives  have  the  potential  to  increase  commitment  and  further  the  development 
and  proficiency  of  their  skills. 

It  is  recommended  that  the  CoP  develop  a  reputation  system  similar  to  Stack 
Overflow,  stackoverflow.com  [17].  A  reputation  system  may  be  implemented  as  a  feature 
on  a  webpage  used  by  the  Navy  CoP  for  programmers  and  developers.  As  demonstrated 
by  Stack  Overflow  users,  reputation  points  may  encourage  members  to  take  a  more  active 
role  to  improving  their  practice  and  that  of  others.  Furthermore,  the  importance  and  value 
benefited  from  focusing  the  design  of  a  CoP  on  competition  and  community  may  further 
increase  active  participation.  Additionally,  reputation  points  or  awards  may  be 
implemented  through  attendance,  participation,  and  winning  hackathon  events.  A 
reputation  system  is  one  form  of  incentive  that  could  easily  be  adopted  as  a  characteristic 
to  a  Navy  CoP  for  programmers  and  developers,  and  simply  integrated  into  the 
operational  structure  of  the  CoP. 

It  is  recommended  that  Tenth  Fleet  develop  a  repository  of  programmer-  and 
developer-  related  knowledge  and  training  that  is  available  for  Navy  personnel  who  wish 
to  become  more  proficient  in  their  practice.  Further  research  is  needed  to  determine  the 
cost  of  establishing  such  a  repository.  This  repository  could  also  be  featured  on  the  CoP’s 
webpage.  CoP  members  should  eventually  take  ownership  of  this  database  in  order  to 
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share  knowledge  and  create  new  knowledge.  Initially  this  website  should  be  unclassified, 
to  allow  access  to  all  CoP  members.  This  repository  could  highlight  readings  and 
trainings  that  are  particularly  relevant  to  current  cyberspace  operations  or  cybersecurity, 
as  a  means  for  practitioners  to  become  more  proficient. 

c.  Measures  of  Success 

Once  sponsoring  organizations  have  been  established,  a  regularly  scheduled 
quarterly  teleconference  of  all  interested  parties  should  be  established  to  generate  a 
guideline  for  measures  of  success.  These  measures  of  success  will  be  necessary  for  the 
evaluation  of  the  CoP,  and  may  indicate  when  changes  and  adjustments  are  needed  to 
improve  the  design  of  the  CoP.  Determined  measures  of  success  should  be  used  as 
guidelines,  and  can  be  tailored  to  the  event  or  interaction  being  evaluated.  Measures  of 
success  may  include  assessment  checklists,  or  surveys  that  can  be  used  before  and  after 
the  completion  of  social  interaction  events.  It  must  be  reiterated  that  these  should  be 
guidelines  and  not  formal  procedures,  due  to  the  drawbacks  of  using  a  heavy  hand  in  the 
design  of  a  Navy  CoP.  CoPs  have  natural  lifecycles  and  may  need  to  be  left  alone  in 
order  to  flourish.  Often  CoPs  need  minimal  support  and  guidance,  therefore  guidelines 
are  only  recommended  tools  that  can  help  in  determining  the  success  of  the  CoP  but  also 
allow  for  the  natural  development,  growth,  and  design  of  a  CoP  to  take  place  over  time. 

3.  Lifecycle 

It  is  important  to  continue  prototyping  various  methods  of  social  interaction  in 
order  to  find  out  what  does  and  does  not  work  for  designing  a  Navy  CoP  for 
programmers  and  developers.  Prototypes  already  mentioned  include  quarterly  social 
interactions  and  design  events  that  take  advantage  of  the  diverse  mindset  of  individuals 
stationed  and  located  throughout  the  world.  Furthermore,  it  is  recommended  that 
stakeholders  conduct  post-event  assessments  in  order  to  determine  their  level  of  success 
and  influence  in  CoP  promotion  and  development.  Furthermore,  it  is  recommended  that 
semi-annual  meetings  be  held  to  include  the  CoP  sponsor,  champion,  and  stakeholders  in 
order  to  address  the  lifecycle  of  the  CoP,  and  determine  what  stage(s)  need  to  be  revisited 
in  order  to  continue  growth  and  momentum. 


73 


B.  AREAS  FOR  FURTHER  RESEARCH 


This  research  demonstrated  the  need  to  conduct  iterative  prototypes  in  order  to 
design  a  successful  CoP.  Feedback  received  from  HACKtheSKY  participants  reflected  a 
shared  desire  for  establishing  monetary  incentives  for  keeping  programming  and 
developing  skills  of  military  members  proficient.  Possible  further  research  on  this  topic  is 
described  in  the  sections  that  follow. 

1.  Iterative  Prototypes 

Additional  prototypes  are  needed  in  order  to  determine  the  forms  of  social 
interactions  needed  to  build  membership  commitment  and  ensure  the  longevity  of  the 
CoP.  Throughout  the  design  of  a  CoP,  opportunities  will  arise  to  test  prototypes  and  run 
several  iterations  of  each  prototype,  but  as  pointed  out  previously,  this  will  take  time.  As 
CoPs  take  time  to  design  and  cultivate.  These  processes  cannot  be  forced,  and  should 
remain  inclusive,  yet  frequent  enough  to  encourage  and  maintain  interest  and 
participation.  Successful  CoPs  go  through  several  design  iterations  and  any  part  of  the 
design  process  may  or  should  be  revisited  in  order  to  find  what  works  best  for  the  CoP. 
Therefore  it  is  imperative  that  invested  members  of  the  CoP  perform  continuous 
assessments  on  the  progress  of  the  CoP  design  and  determine  which  prototype  to 
implement  in  order  to  design  a  more  sustainable  and  effective  CoP. 

2.  Incentives 

Further  research  should  be  conducted  in  regards  to  the  cost  and  benefits  analysis 

of  implementing  a  monetary  incentive  program  for  programmers  and  developers  to  keep 

their  skills  proficient.  A  monetary  incentive  should  include  yearly  testing  to  initially 

establish  and  then  later  on  evaluate  skill  level  and  proficiency,  similar  to  that  of  DOD 

language  skills.  Furthermore,  policy  would  need  to  be  established  in  order  to  properly 

evaluate  how  skills  are  determined.  One  recommended  self-accessed  adult  skill 

acquisition  scale  is  the  Dreyfus  model,  which  consists  of  five  skill  acquisition  levels: 

“Novice,  Advanced  Beginner,  Competent,  Proficient,  and  Expert”  [80].  The  Dreyfus 

model  was  implemented  in  surveys  for  HACKtheSKY  in  order  for  participants  to  self- 

evaluate  their  skill  levels.  There  is  limited  literature  that  supports  or  discourages  the  use 
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of  this  tool;  however,  it  is  one  method  to  utilize  in  determining  the  skill  level  of  Navy 
programmers  and  developers. 

C.  SUMMARY 

Instead  of  formulating  an  end  goal,  opening  the  discussion  about  a  CoP  was  the 
author’s  end  goal  of  this  research.  Extensive  research  and  literature  review  was 
conducted,  in  addition  to  minimal  prototyping.  What  emerged  from  this  initial  design 
phase  was  a  further  need  to  continue  designing  the  CoP.  We  are  confident  that  such  work 
will  be  beneficial  to  the  formal  and  informal  cyber  workforces  and  the  Navy  at  large. 
This  research  has  provided  recommended  tools  and  processes  for  how  to  design  a 
successful  CoP,  but  leaves  the  potential  for  future  researchers  to  expand  on  prototypes 
and  the  adoption  of  a  CoP  based  on  established  principles  and  policies.  Further  research 
will  increase  the  Navy’s  situational  awareness,  and  inform  decision  makers  in 
determining  how  to  design  an  effective,  successful,  and  sustainable  Navy  CoP  for 
programmers  and  developers. 

This  thesis  research  made  the  initial  assumption  that  the  Navy  would  benefit  from 
a  Navy  CoP  for  programmers  and  developers.  The  research  demonstrated  the  importance 
of  replication  and  implementation  of  key  characteristics  found  in  successful  existing 
communities  when  designing  a  Navy  CoP  for  programmers  and  developers.  It  is 
imperative  at  this  point  in  the  lifecycle  phase  of  designing  a  CoP  to  take  necessary  steps 
and  immediate  actions  in  order  to  establish  a  Navy  CoP  for  programmers  and  developers. 
Both  long  term  and  immediate  recommendations  were  provided  on  how  to  design 
this  CoP. 

When  designing  a  CoP  senior  leaders  need  to  be  mindful  of  the  patience  required 
for  the  cultivation  and  prototyping  needed  for  the  creation  of  a  successful  CoP. 
Furthermore,  the  Navy  should  not  rely  on  excessive  measures  of  success,  and  allow  the 
development  of  a  CoP  to  naturally  happen  over  time.  Fiterature  and  feedback  from 
HACKtheSKY  participants’  echoed  the  same  concern,  that  if  the  Navy  focused  too  much 
on  excessive  metrics,  then  this  would  deteriorate  the  nature  of  the  CoP.  Members  would 
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be  less  likely  to  want  to  be  involved  in  the  CoP  if  they  knew  the  Navy  was  more 
concerned  about  metrics  then  allowing  the  CoP  to  develop  on  its  own  overtime. 

Recommendations  for  the  next  lines  of  effort  in  the  design  of  this  CoP  is 
establishing  an  online  presence  in  the  form  of  a  CoP  website,  determining  and  recruiting 
sponsorship,  promoting  the  community,  and  further  prototyping  in  order  to  determine 
what  does  and  does  not  work.  Upon  completion  of  several  iterations,  the  author  is 
confident  that  the  Navy  will  reap  many  benefits  unknown  at  this  time  that  will  come  from 
expanding  its  boundaries  in  the  domain  and  practice  of  programming  and  developing. 
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APPENDIX  A.  PRE-HACKATHON  SURVEY 


The  pre-hackathon  survey  consisted  of  15  questions.  The  intent  of  this  survey  was 
to  collect  demographic  information  about  individuals  attending  HACKtheSKY,  gain  a 
better  understanding  of  their  self-accessed  individual  skill  levels  as  a  programmer  or 
developer,  and  hear  what  their  expectations  of  the  event  were.  Listed  below  are  the 
results  from  the  pre-hackathon  survey: 

(1)  Question  1:  Male  or  Female? 

•  Ninety  two  percent  of  the  participants  were  male,  and  8%  were  female. 

(2)  Question  2:  Current  Job  Title/Description? 

•  Job  titles  and  descriptions  included  Midshipman,  Intellectual  Property 
Specialist,  Network  Forensics  Technical  Director,  Commanding  Officer, 
Head  of  Data  Science,  Project  Manager,  Developer/Operator,  Operations 
and  Training  Staff  Officer,  Cyber  Operations  Officer,  Commander’s 
Innovation  Group  Member,  Cryptologic  Warfare  Officer,  and  Naval 
Innovation  Advisory  Council  member. 

(3)  Question  3:  If  Active  Duty  Military  or  Reservist:  Rate/Rank  and 
Time  in  Service  (years)? 

•  Fifty  six  percent  of  participants  were  active  duty  military  and  6%  were 
reservist.  Collectively,  the  group  contained  over  193  years  of  military 
service.  The  military  ranks  of  participants  ranged  included  Enlisted  (E-6) 
and  Officer  (01-05). 

(4)  Question  4:  If  Military  Veteran:  Rate/Rank  and  Time  in  Service 
(years)? 

•  Nineteen  percent  of  the  participants  were  military  veterans.  The  highest 
Officer  rank  earned  was  the  pay  grade  of  Commander  (0-5),  and  the 
highest  Enlisted  rank  earned  was  the  pay  grade  of  First  Class  Petty  Officer 
(E-6).  Collectively,  the  military  veterans  had  approximately  45  years  of 
combined  military  service. 

(5)  Question  5:  If  Active  Duty  Military,  Military  Reservist  or  Military 
Veteran  -  Designator  at  entry? 

•  Of  the  56%  military  participants  (active-duty,  reservist  or  veterans),  50% 
were  Cryptologist.  Designators  included  Cryptologic  Technician 
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Networks,  Engineering  Duty  Officer,  Intelligence  Officer,  Surface 
Warfare  Officer,  and  an  Aviator. 

(6)  Question  6:  Do  you  have  any  college  level  degrees  related  to 
Computer  Science,  Engineering,  or  Programmer/Developer  skills?  If 
so,  what  level  of  degree  and  in  what  field? 

•  Thirty  three  percent  of  the  participants  had  college  level  degrees  related  to 
Computer  Science  and/or  Engineering.  Sixty  three  percent  of  all 
participants  reported  having  obtained  a  Bachelor  of  Science  degree  or 
higher.  Degrees  obtained  included:  Mathematics,  General  Science, 
Systems  Engineering,  Mechanical  Engineering,  Information  Technology, 
and  Physics.  Several  individuals  had  Master  level  degrees  in  Astronautical 
and  Aerospace  Engineering,  and  Cyber  Systems  and  Operation.  Two 
participants  had  Doctorates  in  Physics  and  Electrical  Engineering.  Two 
participants  are  currently  pursuing  Bachelor  of  Science  degrees  in 
Cyber/Information  Security  and  Cyber  Operations. 

(7)  Question?:  Do  you  have  any  formal  military  and/or  civilian 
education  related  to  programming/developing?  If  so,  what  type  of 
training? 

•  Eight  percent  of  participants  reported  having  400  hours  of  advanced 
programming  training  and  completing  a  Computer  Network  Operations 
internship.  Collectively  the  group  had  minimal  to  no  formal  military  or 
civilian  education  in  relation  to  programming  and  developing,  aside  from 
those  who  had  or  are  pursuing  cyber  related  degrees.  Most  experience 
came  from  college  class  work  associated  with  a  computer  science  related 
degree. 

(8)  Question  8:  How  do  you  assess  your  current  programmer/developer 
skill  level? 

•  To  define  skill  levels,  the  author  used  the  Dreyfus  model,  a  model  used  to 
rank  and  access  adult  skill  acquisition  [80].  The  Dreyfus  model  includes 
five  skill  levels:  Novice,  Advanced  Beginner,  Competent,  Proficient,  and 
Expert  [80].  There  are  certain  criteria  met  that  defines  an  individual’s  skill 
level,  but  more  often  than  not  individuals  self-define  and  evaluate  their 
own  skill  levels.  Of  the  14  participants  who  completed  this  particular 
survey,  33%  claimed  to  be  a  novice,  7%  advanced  beginning,  26% 
competent,  33%  proficient,  and  no  experts. 

(9)  Question  9:  What  languages  do  you  currently  use  and/or  prefer? 

•  Fifty  percent  of  participants  use  and/or  prefer  Python,  29%  ‘C’,  14% 
JavaScript  and  C++,  and  7%  PHP,  Haskell  and  MATLAB. 
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(10)  Question  10:  Do  you  have  any  experience  or  currently  use  any 
particular  repository  or  collaboration  platform? 

•  Fifty  seven  percent  reported  experience  with  GitHub,  43%  Slack,  and  21% 
SourceForge. 

(11)  Question  11:  Are  you  currently  a  member  of  any  formal  or  informal 
Community  of  Practice?  If  yes,  what  is  the  name  and  nature  of  the 
community? 

•  Thirty  three  percent  of  participants  are  current  members  of  a  CoP.  CoPs 
listed  included:  Naval  Constellation,  Navy  Entrepreneurship,  Cyber 
Warfare  Engineers,  Navy  Uniformed  Software  Development,  Ideation 
CoP  (OSD  Design  Thinking),  Defense  Entrepreneur’s  Forum,  Military 
Writer’s  Guild,  Innovation  Entrepreneurship,  and  the  ATHENA  Project. 

(12)  Question  12:  Is  this  your  first  time  attending  a  hackathon?  If  no, 
what  previous  hackathons  have  you  attended  and  what  was  the 
overall  project? 

•  For  58%  of  participants,  this  was  the  first  hackathon  they  had  ever 
attended.  Some  of  the  participants  (25%)  had  previously  attended  the 
United  State  Naval  Academy’s  Hack  the  Yard  hackathon  earlier  this  year, 
which  consisted  of  various  cyber  and  computer  science  related  projects. 

(13)  Question  13:  Do  you  prefer  to  be  on  a  particular  team,  or  are  you 
open  to  being  paired  with  a  team  of  our  choosing  based  on  skill  sets 
and  languages  in  order  to  encourage  collaboration?  Teams  will 
consist  of  3-5  individuals. 

•  Sixty  seven  percent  of  participants  would  have  preferred  to  be  on  a 
random  team,  and  17%  preferred  a  specific  or  pre-selected  team. 

(14)  Question  14:  What  do  you  expect  to  gain  from  attending  this 
hackathon? 

•  The  two  most  common  answers  received  were  to  meet  like-minded 
individuals  and  social  networking  opportunities.  Other  responses  included: 
improve  skills,  available  learning  opportunities,  building  relationships 
with  national  security  interested  individuals,  networking  opportunities 
external  to  service,  developing  relationships,  exposure  to  software  design 
and  prototyping,  gain  better  knowledge  of  coding,  learning  how  to 
collaborate  in  a  hackathon  environment,  explore  how  hackathons  might  be 
used  to  solve  military  problems,  and  impact  traditional  methods  for 
advancing  naval  capability. 


79 


(15)  Question  15:  If  you  have  any  further  comments,  or  information  that 
you  think  would  be  of  value,  please  share  here. 

•  Participants  provided  no  additional  comments  or  information. 
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APPENDIX  B.  RESEARCH  INTERVIEW  QUESTIONNAIRE 


There  were  two  different  research  interview  questionnaires  were  distributed  to 
consenting  participants.  One  questionnaire  was  designed  for  active  duty  military.  All 
other  questionnaires  were  designed  for  civilian  participants.  Participants  were  provided 
the  survey  that  aligned  with  their  registration  information,  and  whether  they  identified  as 
a  military  or  civilian  participant.  For  those  participants  that  did  not  identify  or  provide 
this  information  during  registration,  they  were  emailed  both  questionnaires  and  asked  to 
fill  to  the  one  more  closely  related  to  their  current  job  and  position. 

Military  Research  Interview  Questionnaire 

The  military  research  interview  questionnaire  consisted  of  15  questions.  There 
were  nine  participants  who  responded  to  the  questions  in  this  survey.  The  intent  of  this 
survey  was  to  gather  information  related  to  individual  skills  utilized  within  the  Navy  in 
an  effort  to  better  understand  some  of  the  personable  beliefs  related  to  how  the  Navy  may 
benefit  from  a  CoP  for  programmers  and  developers.  Listed  below  are  the  results  of  all 
questions: 

(1)  Question  1.  Do  you  have  certain  skills  that  aren’t  utilized  in  your 
current  Navy  job?  If  so,  what  are  those  skills? 

•  There  were  three  different  responses  from  the  group:  yes,  no  or  not 
applicable.  One  third  of  the  group  believed  that  their  skills  were  utilized 
effectively.  Another  one  third  of  the  group  stated  that  they  were  too  novice 
in  their  current  position  to  know  if  their  current  skill  set(s)  were  properly 
utilized  in  their  current  position.  One  participant  stated  that  the  majority  of 
their  skills  were  utilized  because  they  forced  them  to  be  utilized.  The 
participant  expressed  concern  that  they  had  been  told  previously  that 
developers  did  not  exist  in  their  current  job,  but  from  their  experience 
developer  skills  have  been  a  crucial  part  of  their  organization’s  operational 
success. 

•  Some  of  the  skills  that  participants  listed  as  being  utilized  in  their  current 
Navy  job  included:  programming,  software  design  and  development,  team 
and  small  group  leadership,  project  management,  organizing  groups, 
rapidly  executing  programs,  expanding  collaboration  opportunities,  and  in 
depth  knowledge  of  institutional  innovation.  One  participant  expressed 
concern  that  even  though  they  were  extremely  skilled  and  talented,  their 
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junior  rank  affected  their  ability  to  be  effective  at  utilizing  those  skills 
under  their  own  cognizance. 

(2)  Question  2:  As  a  programmer/developer,  how  do  you  keep  your  skills 
proficient? 

•  Participants  provided  a  variety  of  answers  to  this  question  ranging  from 
regular  use  and  self-initiative  to  no  effort  because  it  would  affect  their 
family  time.  Some  participants  invested  in  personal  projects  to  learn  new 
skills,  others  chose  to  improve  their  skills  through  academic  course  work. 
The  most  common  answer  provided  was  continual  projects  and  application 
of  skills.  Because  these  skills  are  highly  perishable,  they  must  be 
continuously  utilized  to  maintain  the  level  of  expertise.  One  participant 
located  conferences  to  attend  in  order  to  improve  their  operational 
prowess.  Some  of  the  participants  were  not  programmers  or  developers, 
and  therefore  did  not  provide  additional  details  for  this  question. 

(3)  Question  3:  Do  you  think  it  is  necessary  to  have  skilled 
programmers/developers  in  the  Active  Duty  Navy,  or  could  Navy 
civilian  employees  or  contractors  better  fill  these  jobs?  Why? 

•  One  hundred  percent  of  participants  agreed  that  it  is  both  necessary  and 
extremely  important  for  the  Navy  to  have  active  duty  programmers  and 
developers.  One  argument  made  was  that  active  duty  programmers  and 
developers  bring  a  different  perspective  that  would  not  be  present  in  the 
civilian  or  contractor  culture,  and  that  this  was  especially  true  when 
considering  who  will  lead  developer  teams  working  on  Navy  projects. 
Active  duty  would  allow  for  greater  knowledge  and  greater  ease  in  making 
possible  changes  quickly  to  networks.  The  participant  continued  to  state 
that,  “The  technical  leadership  required  to  manage  these  teams  of 
developers  is  a  different  skill  set  than  required  to  manage  other  military 
fields.  The  developers  (civilian,  military  or  contractor)  will  not  be  as 
responsive  to  leaders  who  have  not  worked  development  projects.  Leaders 
of  developers  need  to  have  been  developers  or  at  the  very  least  need  to 
understand  the  constraints  and  challenges  faced  by  the  developers.” 
[Anonymous].  Another  participant  made  a  similar  argument  related  to  this 
statement  that  sated  everyone  should  learn  some  basic  programming  skills. 

•  Another  participant  agreed  that  it  was  100%  necessary  to  have  active  duty 
military  programmers  and  developers  in  the  Navy,  but  also  stated  that  a 
blend  was  preferable.  A  blend  would  retain  continuity  through  Navy 
civilians,  but  incorporate  the  creativity  and  fresh  points  of  view  from 
active  duty  military  personnel.  They  further  went  on  to  state  that  a  blend 
would  allow  for  professional  development  opportunities  for  active  duty 
personnel  that  they  might  not  otherwise  receive,  and  it  may  deepen  their 
technical  understanding  of  their  job.  Another  great  point  brought  up  by  a 


82 


participant,  was  that  in  the  event  of  a  furlough  or  contract  lapse,  active 
duty  personnel  could  carry  the  load  until  issues  were  resolved,  which  has 
happened  in  previous  years. 

•  Another  participant  supported  the  need  for  active  duty  programmers  and 
developers,  but  stated  that  there  needed  to  be  a  small  cadre  of 
professionals  familiar  with  more  technical  backgrounds.  They  argued  that 
this  small  cadre  of  programmers  should  be  “familiar  with  the  underlying 
mutable  logical  structures  and  code  associated  with  naval  and  maritime 
systems  who  are  also  tasked  directly  with  naval  warfighting.  While  other 
groups  can  augment  and  support,  if  we  are  going  to  realize  any  degree  of 
collective  competence  in  naval  cyber  warfare,  there  will  have  to  be  some 
that  are  experts  in  the  underlying,  technical  cyber  terrain  that  is  largely  a 
composite  of  code  and  purpose-built  computer  hardware.”  [Anonymous]. 
Another  participant  supported  this  belief  and  added  that  they  thought 
certain  rates  and  restricted  line  officers  should  have  these  skills.  In  order 
to  accomplish  these  tasks  mentioned  above,  the  participant  felt  that  it 
would  almost  certainly  have  to  be  led  by  active  duty  personnel,  because  of 
a  limited  commercial  or  non-naval  function  for  this  type  of  expertise. 

•  Lastly,  another  argument  for  why  there  needs  to  be  active  duty  Navy 
personnel  with  programming  and  developing  skills,  was  because  of  their 
title  10  authorities,  which  civilians  do  not  have.  They  also  agreed  that 
there  should  be  a  blend  of  active-duty  personnel  and  civilians  (for 
continuity),  as  well  as  contractors  focused  on  expertise  and  surge  capacity. 
In  short,  there  is  a  need  for  all  three  sectors  to  work  together  toward 
accomplishing  Navy  goals  and  missions  as  related  to  programming  and 
developing. 

(4)  Question  4:  If  the  Navy  paid  you  to  maintain  your 

programmer/developer  skills,  would  that  be  an  incentive  to  stay  in  the 
Navy  and  keep  your  skills  proficient?  Why? 

•  Fifty  five  percent  of  participants  agreed  that  incentive  pay  would  help 
retain  Navy  personnel  as  well  as  encourage  them  to  keep  their  skills 
proficient.  Twenty  seven  percent  of  participants  said  that  incentive  pay 
would  encourage  them  personally  to  maintain  their  programmer  or 
developer  skills.  Thirty  three  percent  of  participants  said  that  it  would  not 
be  an  incentive  for  them  personally.  Of  the  responses  that  supported 
incentive  pay,  participants  agreed  that  a  decent  level  language  pay  would 
potentially  close  the  gap  between  what  industry  pays  versus  the  military. 
A  common  expressed  concern  among  participants  was  their  personal 
experiences  in  watching  talented  active  duty  members  leave  the  Navy  to 
work  for  industry  because  the  pay  was  more  attractive. 
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•  While  incentive  pay  may  go  a  long  way  toward  retaining  active-duty 
personnel,  another  idea  was  to  provide  continual  high  quality  training. 
Another  suggestion  was  the  need  for  both  intrinsic  and  extrinsic 
motivators  (no  further  details  on  what  those  motivators  would  be  were 
given).  One  participant  expressed  concern  that  the  skills  of  programmers 
and  developers  are  not  recognized  in  a  meaningful  way  and  are  not 
formally  supported  by  the  Navy.  They  recommended  that  any  shift  in 
direction  of  recognition  and  support,  pay  or  training,  would  fundamentally 
not  only  change  their  personal  relationship  with  the  Navy  in  a  positive 
way,  but  also  that  of  many  other  active  duty  service  members. 

•  For  those  who  said  that  incentive  pay  would  not  be  an  incentive  to  stay, 
one  reason  given  was  that  they  already  receive  plenty  of  incentives  from 
the  Navy,  such  as  basic  allowance  for  housing.  For  them,  they  were  in  the 
Navy  due  to  a  sense  of  service,  and  they  keep  their  skills  proficient  in 
order  to  help  the  nation  and  do  something  important,  not  for  incentive 
purposes.  Another  participant  was  in  support  of  incentive  pay,  but  said 
that  given  the  number  of  years  they  already  have  invested  in  the  Navy,  that 
the  financial  benefits  of  retirement  were  more  of  an  incentive  than  any 
bonuses  associated  to  a  certain  skill  set.  They  continued  to  state  though 
that  for  Junior  Officers  or  Enlisted  personnel,  that  financial  incentive 
might  encourage  them  to  stay  until  ten  plus  years,  or  even  possibly  a  full 
20  years  of  service. 

(5)  Question  5:  How  do  you  currently  apply  your  programmer/developer 
skills  to  better  yourself? 

•  Fifty  percent  of  participants  stated  that  they  did  apply  their  skills  to  better 
themselves,  while  the  other  half  did  not.  Of  those  who  did  apply  their 
skills  to  better  themselves,  most  of  the  participants  did  so  through  building 
tools  to  solve  their  own  problems,  or  working  on  small  personal  projects 
during  their  own  time.  Building  tools  for  personal  use  allows  one 
participant  to  be  more  self-sufficient,  and  feel  empowered  by  knowing  that 
they  could  build  a  program  that  can  do  nearly  anything  they  wanted  it  to 
do. 

•  Another  participant  viewed  programming  and  developing  skills  similar  to 
that  of  any  second  or  third  verbal  language,  in  that  it  is  a  skill  they  enjoy 
keeping  current  and  they  believed  that  it  makes  them  a  more  valuable 
Naval  Officer.  Another  participant  dedicates  their  time  to  mentoring  others 
who  have  the  time  to  invest  themselves  in  more  actively  larger-scale 
efforts,  but  also  practices  on  small  projects  at  home  or  outside  of  work. 
Lastly,  one  participant  found  benefits  in  automating  a  lot  of  the  manpower 
intensive  work,  allowing  them  to  dedicate  more  time  to  improve  and 
develop  their  own  personal  skills. 
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(6)  Question  6:  How  do  you  currently  apply  your  programmer/developer 
skills  to  better  the  Navy? 

•  Seventy  five  percent  of  participants  attend  conferences,  hackathons  or 
internships  as  a  means  to  apply  their  skills  to  better  the  Navy.  Eight 
percent  of  participants  work  in  technical  fields  where  they  must  know  how 
systems  work,  and  consequently  what  is  done  to  their  individual 
components  in  order  to  find  vulnerabilities.  Furthermore,  these 
participants  must  understand  both  the  computer  and  software  engineering 
components.  While  there  is  a  small  group  of  experienced  and  trained 
Naval  Officers  who  work  at  such  a  technical  level,  the  argument  was  made 
by  one  participant  that  the  Navy  needs  Officers  who  can  do  technical  work 
and  lead  teams  of  people  to  allow  us  to  produce  cyber  capabilities. 

•  Another  participant  stated  that  they  lead  various  cybersecurity-related 
efforts  that  tend  to  be  experimental  or  innovative.  Most  of  these  efforts 
rely  on  a  solid  foundation  of  software  development  principles  and 
application.  Furthermore,  participants  participate  in  events  where  they 
social  network  and  share  knowledge. 

(7)  Question?:  How  do  you  rate  or  evaluate  the  skill  level  of  a 
programmer/  developer? 

•  Some  participants  provided  what  they  would  rate  their  skill  levels  to  be, 
which  included  advanced  beginner,  competent  and  intermediate.  Other 
participants  responded  with  how  they  personally  evaluate  other’s  skill 
level.  One  participant  evaluated  others  based  on  conversations  with  each 
developer.  Another  participant  evaluated  the  speed  and  how  efficient  the 
code  both  looked  and  ran.  Other  participants  evaluated  skills  based  on  the 
combination  of  familiarity  with  various  types  of  programming  syntax, 
critical  thinking  ability,  knowledge  of  how  their  programming/developing 
will  affect  other  systems  in  a  larger  web  of  systems,  and  their  ability  to 
work  in  small  teams. 

(8)  Question  8:  How  important  do  you  think  it  is  for  programmers  and 
developers  to  collaborate  and  network? 

•  Seventy  five  percent  of  the  participants  believed  that  networking  and 
collaboration  was  extremely,  very  important,  critical,  and  absolutely  vital. 
They  agreed  that  it  was  critical  to  the  success  of  the  Navy’s  formal  cyber 
community.  Given  that  there  are  so  few  programmers  and  developers 
within  the  active-duty  ranks,  the  argument  was  made  for  the  need  of  a 
tight  community  that  can  leverage  different  individuals  experience  and 
expertise.  This  community  of  programmers  and  developers  could  be  used 
to  help  active  duty  personnel  on  a  range  of  Navy  platforms,  both  software 
and  hardware,  where  they  might  not  have  the  knowledge  needed  to 
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perform  certain  functions.  Most  participants  agreed  that  networking  and 
collaborating  with  other  programmers  and  developers  is  a  primary  source 
for  better  practices,  new  techniques  and  code  libraries. 

(9)  Question  9:  Would  you  be  interested  in  being  part  of  a  Navy  Cyber 
Community  of  Practice  for  developers  and  programmers?  Why? 

•  Seventy  five  percent  of  participants  were  interested  in  being  part  of  a 
Navy  CoP  for  programmers  and  developers.  Some  participants  wanted  to 
be  a  member  in  order  to  have  a  community  they  could  reach  out  to  when 
running  into  issues  or  questions.  Other  participants  stated  it  would  expose 
them  to  more  challenges,  and  provide  them  and  the  Navy  an  awesome 
resource  for  leveraging  expertise  to  answer  questions.  The  CoP  could  help 
individuals  learn  new  practices  and  develop  professionally.  One 
participant  stated  that  the  Navy  needed  a  CoP  to  allow  for  greater 
knowledge  and  networking. 

(10)  Question  10:  If  you  were  part  of  a  Navy  Cyber  Community  of 
Practice  for  developers  and  programmers,  what  would  be  your  level 
of  contribution? 

•  One  participant  stated  they  would  contribute  more  as  a  technical  leader 
with  a  broad  perspective  on  what  the  community  is  doing  in  cyber.  While 
none  of  the  participants  felt  that  they  would  contribute  specifically  by 
means  of  coding,  they  did  collectively  agree  that  they  could  help  connect 
people  and  help  decompose  a  problem  to  make  it  easier  to  solve.  Another 
participant  suggested  that  they  would  contribute  their  knowledge  and 
experience,  time  to  answer  questions,  solve  problems,  work  on  projects, 
and  mentor  other  programmers.  Whereas  another  participant  said  they 
could  bring  a  sense  of  operational  relevance  to  the  community. 

(11)  Question  11:  As  a  potential  Navy  Cyber  Community  of  Practice 
member,  how  would  the  community  be  of  value  to  you  as  a  member? 

•  Most  participants  stated  that  their  answer  to  this  question  was  the  same  as 
stated  in  question  11.  Other  participants  elaborated  to  share  that  they 
would  benefit  from  a  CoP  by  getting  answers  to  their  questions,  help  with 
solving  problems,  and  mentorship.  Another  member  stated  this  CoP  would 
be  of  value  to  them  due  to  the  ability  to  form  groups  to  solve  problems  ad 
hoc,  as  well  as  way  to  stay  abreast  of  other  individual’s  accomplishments 
and  efforts.  A  CoP  as  stated  by  several  participants  would  be  of  additional 
value  because  of  the  ability  to  rapidly  share  ideas  and  experiences,  as  well 
as  reach  out  to  find  assistance  and  talent  needed  for  certain  cyber  projects. 
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(12)  Question  12:  If  the  Navy  did  have  a  Community  of  Practice  for 

developers  and  programmers,  how  would  you  envision  this  to  look  or 
operate  (ex:  online  community,  website,  face-to-face  interactions)? 

•  Seventy  five  percent  of  participants  supported  face-to-face  interactions 
and  events.  Thirty  three  percent  of  participants  supported  an  online 
community.  Sixty  percent  of  participants  supported  a  combination  and 
balance  of  face-to-face  interactions  and  events  coupled  with  an  online 
community.  Participants  stated  that  face-to-face  events,  such  as 
HACKtheSKY,  provided  a  venue  for  civilians  to  think  about  military 
problems,  as  well  as  military  members  a  venue  to  interact  with  more 
advanced  skilled  programmers  and  programmers  and  sharpen  their  skills 
or  develop  new  skills  based  on  their  networking  and  interactions. 

•  One  participant  suggested  face-to-face  interactions  on  a  “monthly-ish” 
schedule,  hosted  by  various  sub-groups  representing  various  technical  and 
functional  areas.  Another  participant  suggested  the  most  compelling 
interaction  would  be  those  events  hosted  by  local  chapters,  which  would 
enable  more  frequent  face-to-face  interactions.  While  another  participant 
suggested  a  Cyber  Task  Force  face-to-face  interaction,  where  groups 
participate  in  cyber  tabletop  discussions  for  new  equipment,  as  well  as 
cyber  exercises  that  allow  access  to  actual  systems. 

•  In  regards  to  an  online  community,  participants  wanted  an  online 
community  that  provided  the  ability  to  ask  questions  and  share  work.  They 
further  suggested  that  a  Navy  developer  stack  exchange  would  be  a  great 
feature.  They  also  believed  that  the  online  community  should  be  both 
military  (active-duty  and  reservists)  and  civilians.  As  argued  before, 
civilians  bring  long  term  perspective  and  continuity,  and  the  military 
brings  the  practical  warfighting  perspective.  One  participant  in  support  of 
a  primarily  online  community  expressed  interest  in  a  secure  and 
authenticated  online  community.  Another  supporter  of  an  online 
community  suggested  that  there  should  be  challenges  and  face-to-face 
events  that  included  a  point  based  ranking  system  much  like  an  online 
game. 

•  Several  participants  referenced  Stack  Overflow  as  one  of  the  best  online 
communities  for  programmers  and  developers.  What  they  liked  about 
Stack  Overflow  was  that  they  claimed  it  provided  solutions  to  coding  and 
other  issues  that  allow  a  forum  for  help  and  assistance.  They  also  found 
the  website  valuable  for  handling  coding  segments  that  they  found 
challenging,  but  were  solvable  with  the  help  and  review  of  the  code  by 
other  peers.  Most  importantly,  Stack  Overflow,  as  expressed  by 
participants,  was  seen  as  a  one-stop  shop  for  most  of  their  answers  to 
anything  technology  related. 
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(13)  Question  13:  Do  you  think  a  Community  of  Practice  for  developers 
and  programmers  would  serve  a  purpose  and  benefit  for  the  Navy? 

•  Eighty  eight  percent  of  participants  said  the  Navy  would  benefit,  and  one 
participant  said  it  would  depend  on  the  implementation.  The  one 
participant  who  was  concerned  with  implementation  stated  that, 
“implementation  could  go  from  being  a  voluntary  effective  tool  for 
knowledge  sharing,  networking  and  mentoring  to  mandatory  trainings  that 
have  to  be  complete”  [Anonymous].  For  the  rest  of  the  participants  who 
thought  a  CoP  would  serve  a  purpose  and  benefit  for  the  Navy  their 
reasons  included:  exposure  to  new  problems,  global  impact,  potentially 
save  lives,  connect  people  in  order  to  help  each  other  solve  problems,  help 
approval  and  testing  processes,  assist  in  development  of  community 
projects,  professional  development,  share  knowledge,  and  further  develop 
existing  talent(s).  A  more  technical  case  made  by  one  participant,  was  that 
a  CoP  would  allow  the  Navy  to,  “keep  pace  with  the  development  of  new 
threat  capabilities,  to  upgrade  our  existing  capabilities,  or  to  develop 
altogether  new  and  novel  capabilities  at  a  pace  that  many  approximate  or 
surpass  that  of  threat  actors”  [Anonymous]. 

(14)  Question  14:  If  you  could  maintain  a  20-year  career  in  the  Navy  as  a 
dedicated  Navy  cyber  programmer  or  developer,  would  that  interest 
you?  Why? 

•  Forty  three  percent  of  participants  said  that  a  career  as  a  dedicated  Navy 
cyber  programmer  or  developer  would  be  of  interest  to  them,  57%  said  it 
would  not  be  of  interest,  but  that  they  saw  the  benefit,  and  43%  said  it 
would  not  be  of  interest  at  all.  One  interested  participant  would  be 
interested  so  long  the  career  provided  operational  experience  reverent  to 
the  projects  one  would  work  on,  in  addition  to  a  monetary  incentive  close 
to  that  of  what  industry  provides.  Those  participants  who  were  not 
interested  stated  it  was  due  to  the  fact  that  they  desire  to  have  greater 
leadership  experience  instead  of  a  technical  career  path.  While  not 
everyone  would  be  interested,  all  participants  did  agree  that  it  was 
important  to  have  career  pipelines  for  programmers  and  developers. 

(15)  Question  15:  If  you  think  a  cross-Community  of  Practice  is  valuable 
for  the  Navy,  what  sort  of  accommodations  would  the  Navy  need  to 
make  in  order  for  you  to  participate  given  your  current  and  next 
expected  billet  or  assignment? 

•  Some  accommodations  recommended  by  participants  included  setting  up 
an  online  community,  funded  travel  to  community  related  events,  funded 
training,  conference  participation  credit,  increased  promotion  bonuses, 
prizes  for  winning  competitions,  and  the  ability  to  work  real-world 
problems  and  systems.  Some  participants  argued  that  active  duty  members 
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need  time  to  develop  and  work  on  their  programming  and  developing 
skills  without  sacrificing  future  advancement  opportunities.  They  further 
stated  that  the  Navy  needs,  “to  make  programming  and  development  the 
main  job,  not  just  a  job  you  do  on  the  side”  [Anonymous].  Another 
participant  recommended  that  the  Navy  should  allow  the  time  and  ability 
to  build,  maintain,  and  grow  teams  over  time. 

•  One  participant  recommended  that  multiple  locations  should  be  considered 
for  meeting,  including  fleet  concentrated  areas.  This  would  allow  for  easy 
travel  for  more  military  personnel.  One  suggestion  was  one  event  on  the 
east  coast  and  another  on  the  west  coast,  with  the  ability  to  possibly  offer  a 
virtual  teleconference  option  for  those  who  could  not  make  the  travel.  This 
would  allow  multiple  abilities  and  avenues  for  max  military  participation. 
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APPENDIX  C.  CIVILIAN  RESEARCH  INTERVIEW 
QUESTIONNAIRE 


The  civilian  research  interview  questionnaire  consisted  of  15  questions.  Four 
participants  provided  feedback  for  this  survey.  The  intent  of  this  survey  was  to  gather 
information  about  how  individuals  keep  their  skills  proficient,  if  they  were  prior  military, 
and  if  so  whether  or  not  they  utilized  their  programmer  or  developer  skills  while  in  the 
military,  their  experience  with  CoPs,  and  specifically  their  thoughts  on  a  Navy  CoP  for 
programmers  and  developers.  Listed  below  are  the  results  of  all  questions: 

(1)  Question  1:  Prior  to  your  current  job,  did  you  ever  serve  in  the 
military?  If  so,  what  branch,  and  what  was  your  rate/rank/job? 

•  Seventy  five  percent  of  participants  had  prior  military  experience.  One 
participant  served  26  years  total  active  duty  as  an  officer  in  both  the 
United  States  Marine  Corps  and  the  United  States  Air  Force  (13  years  in 
each  respective  service).  The  other  two  participants  served  six  and  nine 
and  a  half  years  respectively  enlisted  in  the  Navy,  in  the  Cryptologic 
Technician  Networks  rating. 

(2)  Question  2:  While  in  the  military,  did  you  perform 
programmer/developer  duties?  If  so,  to  what  extent? 

•  The  prior  military  officer  did  not  perform  programming  or  developer 
duties  while  serving  in  the  military.  One  of  the  prior  enlisted  participants 
served  four  of  their  six  years  as  a  programmer  and  developer.  The 
participant  built  tools  that  ran  on  a  sensor  grid,  wrote  helper  scripts  for 
forensics  personnel,  and  later  wrote  stuff  to  warehouse  and  access  loads  of 
data.  The  other  prior  enlisted  participant  helped  build  and  maintain  a 
custom  sensor  grid  for  their  organization  utilizing  various  programming 
languages.  Additionally,  that  participant  held  various  job  titles  including: 
Forensics  Lab  Manager,  Reverse  Engineer  of  Malware,  Network  Forensics 
and  Packet  Capture  Subject  Matter  Expert. 

(3)  Question  3:  What  did  you  like  most  about  your  work  in  the  military? 
What  did  you  like  least? 

•  What  the  prior  military  officer  liked  most  about  their  work  in  the  military 
was  their  time  in  the  United  States  Marine  Corps  as  a  Company  Grade 
Officer,  and  being  on  the  flight  line  making  airplanes  meet  the  flight 
schedule.  What  the  participant  liked  most  about  being  in  the  Air  Force  was 
the  time  spent  mentoring  young  Company  Grade  Officers  and  filling  the 
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position  of  a  systems  engineer.  The  participant  did  not  share  what  they 
liked  least  about  their  time  in  the  military. 

•  What  one  prior  enlisted  participant  liked  most  about  their  job  in  the  Navy 
was  the  fact  that  they  were  getting  paid  to  code  with  no  previous  academic 
credentials  or  background.  What  they  liked  least  was  the  “utter  lack  of 
commitment  to  meaningful  success”  [Anonymous].  Additionally,  they 
disliked  the  Enlisted  evaluation  process.  They  felt  that  the  evaluation 
process  was  more  focused  on  “check  boxes  and  numbers  vice  incremental 
progress  toward  a  defined  goal  [on]  overall  improvement.  Development  is 
a  continuous  process  where  the  amount  of  work  does  not  necessarily 
immediately  translate  to  a  number”  [Anonymous].  While  this  Sailor  was  a 
highly  rated  performer  at  both  Commands  they  served  at,  they  often  found 
themselves  performing  under  the  mindset  that  is  was  better  do  work  that 
would  “enable”  and  “improve”  processes  instead  of  quantitative 
measurable  work. 

•  What  the  other  prior  enlisted  participant  liked  the  most  about  their  job  in 
the  Navy  was  the  mission  and  the  ability  to  make  a  difference.  What  they 
liked  least  was  the  “bureaucracy,  red  tape,  and  the  lack  of  understanding” 
[Anonymous].  For  them  personally,  they  were  often  frustrated  that  their 
leaders  did  not  understand  or  grasp  the  technical  aspect  of  their  job, 
therefore  often  leaders  squashed  certain  projects  or  ideas.  This  frustrated 
the  participant,  because  a  few  years  later  after  presenting  an  idea  or  project 
to  a  leader  who  later  squashed  it,  they  would  see  a  private  company 
making  millions  off  similar  thoughts  and  ideas. 

(4)  Question  4:  What  led  you  to  your  current  job? 

•  The  prior  military  officer  participant  retired  from  the  Air  Force,  and 
became  a  professor,  since  teaching  was  one  of  their  long-term  desires. 
Another  participant  was  provided  the  opportunity  to  work  at  the 
intersection  of  legal,  data  science  and  coding  fields. 

•  While  still  enlisted  in  the  Navy,  one  participant  was  working  on  a  project 
with  his  current  civilian  boss.  When  that  boss  caught  wind  that  the 
participant  was  getting  out  of  the  Navy,  the  boss  contacted  the  participant 
and  hired  them.  The  other  participant  still  does  the  same  job(s)  as  when 
they  were  enlisted  in  the  Navy,  but  now  they  get  paid  what  they  think  they 
deserve. 

(5)  Question  5:  As  a  programme r/developer,  how  do  you  keep  your  skills 
proficient? 

•  To  keep  their  current  programming  and  developing  skills  proficient,  all 
participants  claimed  to  either  self-teach  or  learn  through  an  academic 
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setting.  One  individual  self-teaches  at  work  by  reviewing  internal 
documentation  and  studying  other  entities  code  changes. 

•  One  prior  enlisted  participant  is  constantly  reading  technical  books  related 
to  programming  and  developing.  They  are  on  multiple  mailing  lists,  and 
they  make  an  effort  to  read  all  applicable  materials  that  will  help  further 
their  knowledge  and  skills.  This  participant  is  also  a  member  of  a 
professional  group  that  openly  shares  information.  This  participant  is 
rather  self-motivated  and  actively  seeks  out  people  who  are  more  skilled 
in  the  area  in  order  to  collaborate.  The  other  prior  enlisted  participant 
claims  that  they  constantly  work  with  like-minded  individuals  and 
thinking  outside  of  the  box  in  order  to  complete  projects. 

(6)  Question  6:  How  do  you  rate  or  evaluate  the  skill  level  of  a 
programmer/developer? 

•  Two  participants  claimed  to  be  a  novice,  while  the  other  two  did  not 
identify  with  a  specific  Dreyfus  skill  acquisition.  Instead,  they  commented 
that  their  skills  are  “based  on  [the]  ability  to  produce  code  with  good 
architecture  split  into  logical  bits  for  code  review  instead  of  one  giant 
blob”  [Anonymous].  One  novice  individual  stated  that  they,  “always 
viewed  programming  as  a  means  to  an  end,  just  like  a  computer  is  just  a 
tool.  I  focus  more  on  the  critical  thinking  and  less  on  the  specific  tool 
used,”  [Anonymous].  Another  self- acclaimed  novice  commented  that  even 
though  they  claim  to  be  a  novice  other  individuals  view  them  as  more 
skilled  than  they  personally  view  themselves. 

•  One  prior  enlisted  participant  rates  and  evaluates  others’  skill  levels  based 
on  their  ability  to  problem  solve  and  deliver.  They  personally  have  no 
faith  in  tests  or  questions,  as  they  tend  to  boil  down  to  trivia.  They  value 
how  an  individual  uses  what  tools  to  tackle  a  problem  as  well  as  their 
ability  to  work  practical  examples.  Often,  the  participant  has  hired 
individuals  that  failed  their  interview  coding  challenge,  but  demonstrated 
solid  logical  approaches  to  problems  presented  within  an  interview. 

(7)  Question  7:  How  important  do  you  think  it  is  for  programmers  and 
developers  to  collaborate  and  network? 

•  All  participants  agreed  that  it  is  very  important  for  programmers  and 
developers  to  collaborate  and  network.  One  participant  has  personally 
witnessed  peers  fail  in  their  field  when  they  were  moved  into  an 
environment  where  their  work  was  audited  or  they  worked  as  part  of  a 
team.  Therefore,  the  participant  takes  value  in  collaboration  and  team 
building  in  order  to  expose  oneself  to  their  flaws  and  improve  skills.  One 
participant  believed  that  if  more  people  would  talk,  issues  would  resolve 
themselves  better  and  faster.  Additionally,  they  believed  that  at  any  given 
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time  there  are  similar  projects  being  worked  on  by  various  individuals 
who  do  not  know  about  the  others’  work,  but  if  they  did  they  could 
collaborate  and  save  time  on  a  project.  Another  participant  made  the  case 
that  open  source  is  a  huge  plus,  and  that  networking  and  collaboration  may 
help  break  the  proprietary  paradigm  [Anonymous]. 

(8)  Question  8:  What  has  or  would  motivate  you  to  join  a  Community  of 
Practice? 

•  All  participants  had  similar  responses  to  this  question.  They  all  agreed  that 
what  would  motivate  them  would  include:  the  ability  to  lean  on  those  that 
really  get  it,  support  in  a  non-intrusive  way,  mentoring,  the  access  to 
people  with  different  experiences  and  areas  of  expertise,  and  the  inherent 
interest  that  comes  with  access  to  learn  from  more  experienced  and  skilled 
individuals. 

(9)  Question  9:  How  do  you  think  you  would  benefit  and/or  contribute  to 
a  Community  or  Practice  of  programmers  and  developers? 

•  One  participant  said  they  would  certainly  benefit  more  than  they  would 
contribute  at  first,  but  that  they  could  contribute  due  to  their  wiser 
perspective  perhaps.  The  other  participants  claimed  they  would  benefit 
from  seeing  how  others  break  down  and  tackle  problems  and  the  social 
collaboration  aspect.  They  claimed  that  a  CoP  would  help  expand  their 
professional  network  and  enable  access  to  jobs  they  might  not  otherwise 
have.  They  also  claimed  they  could  contribute  by  sharing  personal 
experience  on  how  they  were  able  to  learn  to  code  with  no  formal 
academic  background. 

•  Another  participant  believes  that  with  a  CoP,  they  could  globally  solve 
issues  that  are  currently  be  dealt  with.  They  encouraged  the  benefits  of 
opening  community  participation  to  junior  personnel  in  order  to  take 
advantage  of  their  fresh  ideas.  They  also  shared  a  desire  for  the  Navy  to 
host  their  own  GitHub  lab  or  server  to  maintain  this  type  of  collaboration. 

(10)  Question  10:  Do  you  think  it  is  necessary  to  have  skilled 
programmers/developers  in  the  Navy?  Why? 

•  All  participants  agreed  that  it  is  necessary  to  have  skilled  programmers 
and  developers  in  the  Navy.  One  participant  gave  two  reasons  as  to  why 
they  thought  the  Navy  needs  Sailors  who  can  actually  program,  “1.  It  is 
good  for  government  to  have  organic  capability  to  do  things  on  its  own 
and  not  be  beholden  to  contractors.  2.  We  need  educated  Officers  who 
understand  how  to  roll  their  sleeves  up  and  do  it,  in  order  to  better 
administer  $M  [million  dollar]  contracts  as  the  government  oversight  on 
large  complicated  IT  projects,”  [Anonymous]. 
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•  One  participant  believes  that  Navy  personnel  know  better  than  any 
outsider  the  data,  domain  and  challenges  that  the  Navy  faces.  Therefore, 
they  believed  that  the  Navy  could  resolve  many  cyber  issues  with  the 
establishment  of  a  CoP  for  programmers  and  developers,  and  do  so 
without  any  red  tape  or  restrictions. 

•  Another  participant  said  that  it  was  very  important  because,  “The  world  is 
rapidly  moving  to  everything  cyber  and  we  must  make  sure  to  keep  up  and 
be  on  the  forefront.  Cyber  vulnerabilities  in  particular  can  be  extremely 
costly  and  having  the  skilled  developers  to  build  and  support  a  multi-layer 
defense  is  therefore  critical,”  [Anonymous].  Another  participant  added 
that  they  have  witnessed  the  benefits  from  actually  doing  the  job.  They 
argued  that  the  Navy  does  different  work  than  a  lot  of  the  rest  of  the 
world,  in  that  they  have  different  expectations  and  requirements. 
Therefore,  naturally  there  should  be  military  professionals  actually 
creating  applicable  solutions. 

(11)  Question  11:  Do  you  think  a  Community  of  Practice  for  developers 
and  programmers  would  serve  a  purpose  and  benefit  for  the  Navy? 
Why? 

•  All  participants  agreed  that  a  CoP  for  developers  and  programmers  would 
“possibly”  serve  a  purpose  or  benefit  for  the  Navy.  One  participant 
suggested  that  there  would  have  to  be  boundaries  placed  on  the  CoP  in 
order  for  it  to  be  productive  in  generating  something  of  relevance  to  the 
Navy.  They  went  on  to  state  that  keeping  cyber  skills  sharp  would  produce 
a  cadre  of  cyber  warriors,  but  boundaries  may  need  to  be  put  into  place  to 
harness  the  enthusiasm  and  channel  it  toward  mission  support.  They  also 
believed  that  while  CoPs  should  be  something  fun  and  constructive  with 
possible  incentives  to  compete  against  other  teams  and  win  cash  prizes, 
the  goal  of  events  should  ultimately  be  to  support  Navy  mission(s).  If 
boundaries  weren’t  put  in  place,  the  possibility  of  a  CoP  becoming  a 
“shadow  organization”  that  works  in  multiple  directions  will  counter 
where  the  overall  mission  is  headed.  Additionally,  one  participant  believed 
that  due  to  the  Navy’s  access  to  many  datasets,  that  the  Navy  could  help 
create  more  data  science  and  analytics. 

•  Another  participant  suggested  that  the  Navy  might  benefit  from  learning 
circles,  and  how  they  help  motivate  individuals,  especially  those  who 
possibly  doubt  their  skill  levels  and  are  apt  to  failing  without  mentorship. 
Another  participant  backed  this  up,  by  stating  that  they  have  seen  a  lot  of 
people  in  their  situation,  where  they  are  smart  individuals  but  have 
essentially  no  technical  talent,  but  they  are  expected  to  perform  a  role  as  a 
programmer  and  developer.  Therefore,  they  are  left  to  explore  on  their 
own  and  become  self-taught  programmers  or  developers.  The  purpose  of  a 
Navy  CoP  for  programmers  and  developers  would  be  the  sharing  aspect. 
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In  this  participant’s  experience,  open  collaboration  has  always  helped 
them.  More  often  than  not  many  problems  they  were  working  on  had 
already  been  solved,  but  due  to  the  lack  of  collaboration  with  other 
programmers  and  developers  they  were  not  aware  of  the  progress  of 
individuals  had  already  made. 

•  One  participant  shared  a  personal  experience  about  a  colleague  who  was 
trying  to  build  a  new  thing.  This  colleague  emailed  the  participant.  The 
participant  replied  back  with  a  link  to  an  already  completed  project  that 
related  to  what  the  colleague  was  trying  to  build.  This  collaboration  saved 
the  colleague  months  of  work,  and  as  a  result  the  colleague  ended  up  with 
a  better  solution  then  having  started  from  scratch.  The  participant  further 
stated  that  one  reason  the  Navy  community  needs  collaboration  is  due  to 
its  relatively  small  size  yet  wide  distribution. 

(12)  Question  12:  In  your  experience,  what  are  some  possible  inhibitors 
from  collaboration  with  the  government  and  civilian  sections,  related 
to  programming  and  developing? 

•  One  participant  stated  that  the  industry  is  profit-driven  and  pays  to  have  a 
programming  source  or  outsources  it.  Whereas  the  government  acquires 
programming  expertise  in  order  to  accomplish  the  mission  of  integrating 
tools  for  mission  use.  They  further  went  on  to  state  that  they  did  not  think 
there  was  much  potential  for  true  collaboration  due  to  the  DOD 
acquisition  program. 

•  Another  issue  raised  by  participants  is  bridging  the  gap  on  trust-in-the- 
government  that  currently  exists  in  the  technology  industry.  According  to 
the  participant,  the  technology  culture  is  very  anti- authority  and 
contrarian,  and  that  the  usual  appeals  of  a  CoP  might  be  difficult.  Another 
issue  mentioned  was  the  constraints  put  on  the  Navy,  and  often  called  “red 
tape,”  in  addition  to  a  lack  of  leadership  understanding  cyber  issues  and 
tools,  egos,  and  archaic  thinking. 

•  One  participant  believed  that  Sailors  move  around  too  often,  between  both 
jobs  and  duty  stations.  They  believe  that  this  constant  change  negatively 
impacts  a  Sailors  ability  to  become  an  expert  in  their  practice.  They 
believe  that  introductory  schools,  often  referred  to  in  the  Navy  as  “A” 
schools,  do  not  produce  expert  programmers  or  developers,  instead  they 
teach  them  basic  skills  and  then  it  is  upon  that  individual  to  learn  more  if 
they  desire  to  excel  and  further  their  skills.  But  as  previously  mentioned, 
when  a  Sailor  is  constantly  changing  their  job  or  moving  to  a  new 
location,  this  does  not  afford  them  the  time  to  improve  on  their  skills 
unless  they  do  it  on  their  own  time  outside  of  working  hours. 
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(13)  Question  13:  Would  you  be  interested  in  being  part  of  a  Navy  Cyber 
Community  of  Practice  for  developers  and  programmers,  and  what 
would  be  your  level  of  contribution  to  the  community?  Explain. 

•  Seventy-five  of  participants  would  be  interested  in  being  part  of  a  Navy 
CoP  for  programmers  and  developers.  One  participant  stated  they  would 
not  be  interested  given  their  domain  of  interest  is  not  so  much  cyber,  but 
more  focused  on  data  science,  and  they  did  not  feel  they  would  have  much 
to  contribute.  The  other  participants  would  like  to  be  members,  and  would 
love  to  help  in  any  way  they  could,  skills  permitting.  They  believe  in  the 
mission  of  the  Navy  and  are  eager  to  help  in  any  way  possible.  One 
participant  even  offered  that  as  a  member  of  the  CoP,  and  given  his 
current  job  location,  that  their  command  could  create  scenarios  that 
needed  to  be  worked  on  at  a  future  hackathon  or  similar  event. 

(14)  Question  14:  As  a  potential  Navy  Cyber  Community  of  Practice 
member,  how  would  the  community  be  of  value  to  you  as  a  member? 

•  One  participant  said  they  would  benefit  from  the  awareness  of  issues  that 
were  relevant  and  timely  to  the  community.  Another  participant  said  it 
would  be  great  to  learn  and  teach  others,  and  further  their  personal 
understanding  of  the  Navy,  as  well  as  networking  with  like-minded 
people.  Another  participant  believes  that  they  have  operational  knowledge 
and  lots  of  ideas,  but  little  time  to  complete  them,  therefore  a  forum  would 
help  them  with  voicing  their  thoughts  and  concerns  and  getting  assistance 
from  other  members. 

(15)  Question  15:  If  the  Navy  did  have  a  Community  of  Practice  for 
developers  and  programmers,  how  would  you  envision  this  to  look  or 
operate  (ex:  online  community,  website,  face-to-face  interactions)? 

•  All  participants  suggested  an  online  community  given  the  far-flung  nature 
of  the  Navy,  but  to  also  include  face-to-face  interactions.  One  participant 
recommended  that  an  online  community  should  resemble  the  features 
found  on  Slack  or  Google  group.  They  also  recommended  that  the  online 
community  persistent  and  asynchronous.  Additionally,  participants 
believed  that  face-to-face  interactions  would  help  in  building  personal 
bonds  between  individuals.  Recommendations  were  made  for  virtual 
teleconferences  and  future  hackathons.  One  participant  questioned  the 
classification  level  of  the  community.  One  participant  recommended  a 
open  platform  similar  to  how  Git  operates  to  develop  the  linux  kernel. 
They  also  recommended  selecting  several  collaboration  sites  that  are 
currently  hosted  to  the  public,  and  allowing  members  to  vote  on  their 
preferred  platform. 
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•  One  warning  that  one  participant  addressed  was  the  Navy’s  tendency  to 
get  over-indulged  in  measurable  requirements.  They  expressed  concern 
that  the  Navy  actively  engaged  immediately,  which  to  the  participant  is 
not  only  exhausting,  but  discourages  them  to  be  a  member  of  a  team  with 
leadership  that  treats  them  as  a  tool  and  not  as  a  person. 
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APPENDIX  D.  POST-HACKATHON  SURVEY 


The  post-hackathon  survey  consisted  of  14  questions.  Thirteen  participants 
provided  feedback  for  this  survey.  The  intent  of  this  survey  was  to  gather  feedback  and 
recommendations  on  participant’s  experience  at  HACKtheSKY.  Listed  below  are  the 
results  of  all  questions: 

(1)  Question  1:  Did  you  attend  all  three  days  of  the  hackathon? 

•  Seventy  seven  percent  of  participants  attended  all  three  days  of  the 
hackathon  (Friday  through  Sunday). 

(2)  Question  2:  How  did  you  hear  about  the  hackathon? 

•  Sixty  two  percent  of  participants  heard  about  the  hackathon  from  one  of 
the  HACKtheSKY  event  planners.  Fifteen  percent  heard  about  the 
hackathon  by  word  of  mouth,  15%  through  social  media,  and  8%  through 
an  existing  CoP  that  they  were  members  of. 

(3)  Question  3:  Why  did  you  attend  the  hackathon? 

•  The  most  common  answers  provided  for  why  participants  attended  was  to 
meet  like-minded  individuals  and  network,  increase  knowledge,  and 
improve  programmer  and  developer  skills.  One  participant  thought  the 
event  looked  like  a  great  way  to  give  back  to  their  country  since  it  was 
military  related.  Another  participant  initially  attended  under  the  pretense 
that  they  were  going  to  be  a  mentor  and/or  judge,  and  instead  ended  up 
mentoring  and  participating  in  the  event. 

•  Other  participants  viewed  the  hackathon  as  an  opportunity  to  play  with 
new  frameworks  and  networks  across  various  Communities  of  Interest. 
Some  participants  attended  because  their  Command  paid  for  their  travel. 
One  participant  attended  because  their  boss  told  them  to.  One  participant 
attended  because  they  believed  in  establishing  a  community  of  developers 
within  the  Navy,  as  well  as  working  with  industry  to  gain  insight  into  good 
and  bad  practices. 

(4)  Question  4:  Did  the  hackathon  meet  your  expectations?  Please 
explain. 

•  Eighty  five  percent  of  participants  did  believe  that  the  hackathon  met  their 
expectations.  Fifteen  percent  of  participants  believed  that  the  hackathon 
did  not  meet  their  expectations.  Some  participants  exclaimed  that  the 
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hackathon  exceeded  their  expectations  because  of  the  networking  aspects, 
as  well  as  the  fact  that  they  were  surprised  by  how  much  they  were  able  to 
contribute.  Some  participants  thought  that  they  would  not  be  able  to 
contribute  due  to  their  lack  of  familiarity  with  the  drone  infrastructure,  but 
they  proved  themselves  wrong. 

•  Participants  were  impressed  with  all  the  “great  minds  working  in  teams,” 
and  that  overall  they  learned  more  than  expected  from  different  people  on 
their  team  as  well  as  other  groups  participating  in  the  hackathon.  One 
participant  stated  that  they  were  pleased  with  the  extensive  networking 
outside  and  inside  of  the  Department  of  Navy,  as  well  as  the  exposure  to 
all  the  methods  and  practical  difficulties  of  both  the  workshop  and 
hackathon.  Several  participants  commented  on  the  “cool  location”  in 
regards  to  where  the  hackathon  was  located  and  hosted  in  San  Francisco. 

•  For  one  participant,  this  was  not  their  first  hackathon.  A  matter  of  fact 
they  had  been  to  several  hackathons,  but  they  stated  that  this  was  the  best- 
organized  one  they  had  attended.  They  appreciated  the  clearly  outlined 
problems.  One  recommendation  they  had  was  that  the  code  be  shared 
further  in  advance  instead  of  the  day  of  the  event.  They  believed  that  if 
they  had  had  the  code  prior  to  attending  the  event,  then  they  could  have 
developed  a  better  code  to  solve  the  problem. 

•  One  participant  recommended  that  the  event  would  have  been  better  if 
there  were  a  fourth  day  in  order  for  hackers  and  design  thinkers  to  work 
together  on  a  team  in  order  to  better  implement  stronger  solutions  for 
presentation.  Several  participants  had  hoped  that  there  would  have  been 
more  adversary  focused  problem  sets  where  they  would  “break  things,”  as 
opposed  to  cybersecurity  problem  sets  and  “creating  fixes.”  While  those 
expectations  were  not  met  for  some  participants,  they  did  benefit  from 
coding  and  developing  “Proof  of  Concept”  code  solutions. 

(5)  Question  5:  What  did  you  learn  during  the  hackathon? 

•  For  most  participants  this  was  their  first  time  working  with  the  drone 
Remote  Operating  System.  For  some  participants  this  was  their  first  time 
working  with  service  members,  and  it  gave  them  a  better  understanding 
about  how  the  cyber  community  is  structured  within  the  military.  One 
participant  stated  that  they  were  surprised  with  how  many  people  could 
code  within  the  Navy  than  they  initially  thought.  They  also  learned  that 
‘we  all  work  in  silos  and  we  need  to  be  working  across  organizations, 
because  there  are  [more]  coders  than  I  thought,  but  not  enough  to  cover  all 
[of]  the  Navy’s  problems”  [Anonymous].  Other  participants  discovered 
through  attending  the  hackathon  and  working  on  small  teams  that  they 
needed  to  enhance  their  coding  skills  and  exposure  to  different 
programming  languages. 
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•  For  the  participants  that  attended  the  Design  Workshop,  they  learned  more 
about  facilitation,  design  thinking  skills,  and  human  based  design  planning 
methods.  One  participant  realized  that  they  needed  to  incorporate  and  train 
on  more  design  methodologies,  as  they  would  greatly  benefit  any  project  a 
team  is  working  on. 

(6)  Question  6:  What  did  you  most/least  like  about  the  hackathon? 

•  Participants  stated  that  meeting  other  participants  was  the  best  part  of  the 
hackathon.  One  civilian  participant  liked  the  fact  that  they  were  able  to 
meet  military  individuals  working  on  cyber  issues.  One  participant  who 
just  recently  joined  the  reserves  was  most  excited  about  discovering  that 
they  would  apply  their  programming  skills  in  the  Navy,  as  they  had  fears 
that  they  would  not  be  able  to  apply  them  fully  in  the  Navy.  This 
hackathon  showed  them  that  there  were  more  opportunities  to  use  their 
programming  skills  than  initially  thought  when  they  first  joined. 
Therefore,  they  felt  a  new  sense  of  excitement  about  joining  the  Navy. 

•  Other  participants  enjoyed  the  diversity  of  both  participants  and  problem 
sets  presented  for  both  the  hackathon  and  design  workshop.  Participants 
who  attended  the  design  workshop  thought  that  the  design  methodology 
phase  was  “enlightening.” 

•  One  participant  stated  that  what  they  liked  least  was  the  distance  they  had 
to  travel  each  day,  due  to  the  location  of  the  event  and  where  they  lived 
locally.  One  participant  wished  that  one  of  the  hackathon  prompts 
(multicast)  were  more  challenging.  Other  participants  wished  they  had  had 
more  time  to  socialize  with  their  teams.  They  felt  that  they  were  so 
focused  on  their  specific  problem,  that  they  did  not  take  the  time  to  talk  to 
other  individuals  as  much  as  they  would  have  liked  throughout  the 
weekend.  Other  feedback  regarding  things  participant  least  liked  included: 
too  early  of  a  start  time  on  Saturday  and  Sunday,  better  snacks  and  drinks, 
more  information  about  team  formation  strategies,  and  lack  of  focus  and 
preparation  on  Friday’s  baseline  briefs. 

(7)  Question  7:  What  recommendations  would  you  make  for  future 
hackathons? 

•  The  number  one  recommendation  was  a  hackathon  that  focused  on  other 
real-world  scenarios  and  issues.  One  recommendation  was  defensive  cyber 
operations.  Other  technical  recommendations  included:  working  out 
technical  setup  kinks,  increased  standard  tool  use  (e.x.  GitHub),  set  up 
virtual  boxes  in  order  to  allow  people  to  interact  with  running  code,  clear 
and  upfront  brief  on  problem  sets,  more  lead  time  on  scope  of  objectives 
and  challenges,  and  earlier  distribution  of  code  base  to  participants  prior  to 
hackathon. 
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•  Non-technical  recommendations  included:  never  running  out  of  coffee, 
including  more  civilians  at  the  event,  explain  to  participants  what  to 
expect  from  attending  their  first  hackathon,  incorporate  hackers  and  design 
thinkers  onto  one  team  as  opposed  to  two  separate  events.  Additionally, 
regarding  the  two  separate  events,  one  participant  recommended  that  the 
team  formation  and  facilitation  process  for  the  design  workshop  should  be 
better  organized  next  time  and  separated  from  the  hackathon.  Lastly,  while 
participants  liked  the  venue,  they  did  not  like  the  chairs. 

(8)  Question  8:  If  there  were  another  Navy  Cyber  Community  of 
Practice  hackathon,  would  you  attend?  Why? 

•  Ninety  two  percent  of  participants  said  they  would  attend  another 
hackathon.  One  participant  said  they  would  not  attend,  because  they  were 
not  a  programmer  or  programmer,  and  were  just  there  for  the  design 
workshop.  Participants  said  they  would  attend  another  one  given  their 
positive  experiences  from  this  recent  event.  They  would  also  attend 
another  one  for  the  same  reasons  they  attended  this  one:  social  networking 
and  enhancing  coding  skills. 

•  Other  reasons  participants  would  attend  another  hackathon  included:  to 
see  “state  of  the  art”  technology,  the  importance  to  continue  building  these 
type  of  relationships  and  share  information,  networking,  refresh  coding 
skills,  cross  Navy  idea  pollination,  broaden  the  experience  of  developers 
and  give  them  viewpoints  on  problem  sets  never  before  considered  by 
most,  maintenance  of  skill  sets,  and  personal  growth.  As  one  participant 
best  stated,  “It  is  also  a  good  opportunity  to  refresh  coding  skills.  It  is  like 
any  other  language  proficiency.  If  you  don’t  use  it,  it  goes  away” 
[Anonymous].  Another  participant  believed  that  both  the  hackathon  and 
CoP  concept  were  valid  and  they  want  to  see  both  evolve  from  this  first 
event. 

•  One  participant  said  they  would  attend,  because  they  had  the  opportunity 
to  meet  like-minded  individuals  who  were  working  on  similar  projects  as 
them  within  their  civilian  organization.  Through  the  weekend  interaction 
and  meet  up,  they  were  able  to  overcome  issues  they  both  had  on  a  similar 
project.  They  were  also  able  to  share  research  and  assist  each  other  on 
their  current  civilian  job  projects. 

(9)  Question  9:  Were  you  on  a  predetermined  team,  or  a  team  of  random 
individuals?  Please  explain  your  experience  in  regards  to  interactions 
and  collaboration  with  team  members. 

•  Seventy  five  percent  of  participants  were  on  a  team  made  of  random 
individuals  and  25%  were  on  a  pre-determined  team,  this  includes  both 
hackathon  and  design  workshop  participants.  Each  participant  had 
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different  experiences  regarding  how  they  met  their  teammates  and 
conducted  collaboration.  One  participant  met  a  group  of  military  veterans 
during  Friday  night’s  social,  after  they  started  socializing  with  the  veterans 
they  were  then  asked  to  join  their  team.  The  participant  was  the  only 
industry  member  on  the  team.  They  worked  well  together  and  enjoyed 
their  experience  so  much  that  they  continue  to  communicate  post- 
hackathon.  Participants  who  were  on  a  team  of  both  military  and  industry 
participants  appreciated  the  range  of  perspectives  and  skills  gained  from 
collaboration. 

•  Some  participants  who  were  on  a  random  team  enjoyed  the  opportunity  to 
self  select  their  teammates.  It  allowed  them  to  find  a  good  balance  of 
personalities  to  work  with.  One  team  was  formed  from  conversations  held 
at  Friday  night’s  social.  What  they  did  not  realize  till  Saturday  morning 
after  continued  conversations  was  they  were  mostly  active  duty  military. 
This  group  formed  a  team  and  collaborated  very  effectively,  split  up  tasks 
and  accomplished  them  quickly.  They  reported  that  they  all  came  together 
and  merged  their  work  into  one  product.  One  participant  from  this  team 
stated  that  “everyone  had  a  great  attitude  and  equal  say  in  each  aspect  of 
the  product  produced”  [Anonymous].  Even  outside  of  the  hackathon  and 
after  hours  they  shared  meals  and  drinks. 

•  One  participant  was  on  a  pre-determined  team  that  consisted  of  them  and  a 
teammate  from  their  industry  job.  They  were  more  interested  in  focusing 
on  the  problem  instead  of  learning  to  work  and  collaborate  with  a  group  of 
random  people  in  a  short  period  of  time.  From  previous  experience  at 
other  hackathons,  they  have  had  mixed  results  from  working  on  random 
teams. 

•  One  design  workshop  participant  was  not  able  to  attend  the  Friday  night 
social  and  therefore  was  randomly  thrown  onto  a  team.  The  difficulty  they 
experienced  was  the  lack  of  time  that  they  had  to  get  to  know  one  another. 
They  felt  that  when  you  do  not  know  the  people  on  your  team  ahead  of 
time,  that  the  “forming,  storming  and  norming”  stages  of  team 
development  get  accelerated  or  skipped,  which  can  become  a  barrier  to 
team  progress  and  success.  One  recommendation  was  to  share  bios 
possibly  for  all  participants  in  order  to  make  teammates  familiar  in  the 
preceding  days  or  weeks  prior  to  an  event  in  order  to  build  trust.  They  also 
recommended  that  event  planners  should  put  more  time  in  to 
acknowledging  new  members,  welcoming  them,  and  providing  guidance 
on  how  to  move  forward  with  projects. 
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(10)  Question  10:  Did  you  expand  your  social  network  due  to  your 
attendance  at  the  hackathon? 

•  All  participants  stated  that  they  expanded  their  social  network  due  to 
attending  HACKtheSKY.  One  participant  reported  connecting  with  over  a 
dozen  new  individuals.  Another  participant  reported  that  they  have 
remained  in  touch  with  teammates  and  other  participants  via  Slack,  email 
and  Facebook.  For  some  of  the  participants  they  had  only  met  attendees 
once  or  twice  before,  or  only  via  email,  so  attending  this  event  was  a  great 
way  for  them  to  put  a  name  to  face  and  further  interact,  converse, 
collaborate,  and  brainstorm  ideas.  Participants  commented  that  the  social 
networking  that  took  place  was  one  of  the  most  important  pieces  of  the 
weekend. 

(11)  Question  11:  Did  you  find  the  hackathon  to  be  of  value?  How  and 
why? 

•  All  participants  believed  the  hackathon  to  be  of  value.  Some  reasons 
provided  that  participants  found  it  to  be  of  value  included:  tangible  results, 
expanding  human  networks,  knowledge  gained,  reviving  rusty  coding 
skills,  and  free  shirts  and  stickers.  One  participant  said  that  the  hackathon 
“opened  my  eyes  to  the  complexity  of  our  system  design  and  the  passion 
of  many  individuals.  I  think  meeting  everyone  on  my  team  and  the 
methods  taught  during  the  workshop  will  help  me  lead  future  efforts  to 
produce  innovate  solutions  to  our  toughest  problems”  [Anonymous]. 
Another  participant  shared  that  they  thought  this  event  was  a  great  start  to 
developing  a  CoP,  and  “that  the  more  we  can  focus  on  tackling  big 
opportunities  and  moving  into  execution/follow-on,  the  more  compelling 
these  events  will  become”  [Anonymous]. 

(12)  Question  12:  Do  you  think  the  Navy  would  benefit  from  future 
hackathons,  if  so,  how  frequent  do  you  think  they  should  be?  (Ex: 
Annual,  bi-annual,  quarterly) 

•  Forty  six  percent  of  participants  recommended  future  hackathons  being 
hosted  bi-annually  (twice  a  year),  31%  voted  for  quarterly  events,  and 
23%  voted  for  a  future  hackathon  to  be  hosted  annually.  Some  participants 
recommended  a  combination  of  quarterly  events,  with  at  least  two  of  those 
events  being  a  hackathon.  Several  participants  stated  that  a  quarterly 
hackathon  might  be  too  often  and  annually  might  be  too  infrequent  and 
that  bi  or  semiannually  would  be  the  sweet  spot.  Participants  also 
suggested  that  frequency  should  depend  on  location,  and  recommended 
that  hackathons  be  geographically  distributed. 

•  One  participant  recommended  trying  quarterly  hackathons  if  the  interest 
persisted  and  if  it  helped  in  keeping  developing  ties  within  the  local 
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community  and  industry.  Another  participant  stated  that  if  the  hackathon 
was  in  the  same  city  every  time,  then  they  would  recommend  it  to  be  bi¬ 
annual  or  annually,  but  if  the  event  moved  from  city  to  city  that  they  could 
occur  more  frequently.  One  participant  suggested  that  an  annual  hackathon 
would  allow  for  enough  time  for  both  preparation  and  developed  interest. 

•  One  participant  stated  that  a  future  hackathon  would  be  “hugely  beneficial 
both  socially  and  to  develop  a  community  within  the  DOD”  [Anonymous]. 
They  also  recommended  that  a  hackathon  be  held  quarterly  or  bi-annually 
in  order  to  provide  ample  opportunity  for  personnel  to  attend  given  busy 
schedules.  Another  participant  acknowledge  the  importance  of  a  regular 
drum  beat,  but  recommended  that  future  hackathons  should  be  centered  on 
opportunities  rather  than  a  specific  schedule.  They  recommended  to  “let 
the  demand  fuel  the  events  more  than  a  schedule.  For  team  hosting  events, 
bi-annual  was  feasible,  quarterly  a  stretch,  and  annual  would  work,  but 
may  not  be  frequent  enough  to  catalyze  the  community”  [Anonymous]. 

(13)  Question  13:  Do  you  think  the  same  effects  could  have  been  achieved 
through  other  means  than  a  face-to-face  hackathon?  Please  explain 

•  Fifty  eight  percent  of  participants  stated  that  the  same  effects  could  not  be 
achieved  through  other  means  than  a  face-to-face  hackathon.  Thirty-six 
percent  of  participants  thought  it  might  be  possible  or  there  were  perhaps 
other  ways,  and  6%  thought  the  same  effects  could  have  been  achieved  in 
other  means.  Most  participants  who  believed  that  the  face-to-face 
hackathon  effects  could  not  have  been  achieved  otherwise  were  mainly 
due  to  the  social  networking  aspect  of  the  weekend.  They  echoed  that  this 
was  a  vital  part  of  the  event.  They  believed  that  putting  faces  with  names 
and  exchanging  business  cards  was  the  most  beneficial  part  of  the  events. 

•  Other  participants  did  not  believe  the  same  effects  could  be  achieved, 
because  there  is  not  a  replacement  for  rapid  development,  face-to-face 
interactions  are  vitally  important,  these  events  build  trust,  and  working  in 
person  produces  better  ideas  and  focused  work  for  a  period  of  time,  just  to 
name  a  few.  One  participant  said  that  any  other  means  of  an  event  would 
not  have  produced  the  same  ‘fun’  feeling  that  HACKtheSKY  did.  For  this 
participant  “seeing  like-minded  folks  in  a  room  crashing  on  a  problem  is 
awesome”  [Anonymous]. 

•  One  participant  believed  that  absolutely  not  the  same  affects  could  have 
been  achieved  through  any  other  means  than  a  face-to-face  hackathon. 
Their  argument  was  that  “the  face-to-face  interactions,  opportunity  for 
formal  and  informal  team-building,  serendipitous  interactions/exchanges, 
and  the  sense  of  urgency  and  focus  that  comes  from  being  in  a  socially 
engaging,  team-oriented  environment  is  without  parallel.  This  could  not  be 
accomplished  without  something  like  this  tangible  event”  [Anonymous]. 
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•  Another  participant  recommended  doing  another  event  a  month  of  two 
after  the  initial  hackathon  to  ensure  “the  camaraderie  and  trust  are  already 
established  by  most  participants”  [Anonymous].  They  also  added  that 
face-to-face  hackathons  should  give  a  lesson  on  distance  collaboration. 

•  Another  participant  recommended  that  doing  a  hybrid  event  would 
possibly  produce  the  same  affects.  Their  recommendation  included  doing 
a  live  event,  where  some  parts  of  it  were  recorded,  such  as  the  opening 
description  of  the  challenges.  This  video  could  then  be  posted  to 
YouTube.  They  further  recommended  coupling  the  event  with  a  link  to 
MMOWGLI,  Slack  and  GitHub,  as  a  way  to  open  it  to  the  rest  of  the 
world  for  participation. 

(14)  Question  14:  Please  include  any  additional  feedback, 

recommendations,  or  thoughts  and  concerns  regarding  your 
participation  during  the  hackathon. 

•  Some  additional  feedback  provided  by  participants  included:  great  facility 
but  parking  was  not  ideal,  time  well  spent,  connect  the  design  thinking 
team  with  hackathon  programmers  and  developers,  and  keep  doing  the 
great  work.  One  participant  recommended  the  use  of  standard  tools  such 
as  GitHub.  They  believed  that  by  using  standard  tools,  primarily  GitHub 
due  to  its  popular  use  in  the  technology  industry,  events  would  run  more 
efficiently  and  smoother. 

•  One  participant  reiterated  the  necessity  to  “get  private  sector  folks 
interested  in  this  kind  of  stuff  and  try  hosting  such  hackathons  more 
frequently”  [Anonymous].  They  further  recommended  that  to  the  extent 
people  request  being  placed  onto  a  random  team,  that  event  planners 
should  try  to  match  industry  and  military  members  together.  This  would 
facilitate  the  private-public  exchange  and  connections.  The  participant 
also  recommended  that  it  would  be  great  if  there  were  a  website  dedicated 
to  team  learning  coupled  with  regular  in-person  meetups.  The  purpose  of 
these  interfaces  and  collaboration  opportunities  would  be  to  “help  folks 
ramp  up  on  various  coding  styles/frameworks  and  grow  in  other’s 
presence”  [Anonymous]. 
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